1st annual conference of the florida association of college test administrators february 3-4, 2010...

1st Annual Conference

of the

Florida Association of College Test Administrators

February 3-4, 2010

Lakeland Florida

Dr. Louis Woodruff

The Florida State Flag

The Flag of the State of South Florida

The oldest continually inhabited city in the

USA is St Augustine, founded in 1565.

The world’s first scheduled passenger

airline flight was on January 1, 1914

from St. Petersburg's to Tampa.

Ybor City was once known as

“The Cigar Capital of the World”

producing 700,000,000 cigars annually

Fort Lauderdale is known as the

Venice of America because it has

185 miles of local waterways.

Venice Florida is known as the

Shark Tooth Capital of the World.

The Moonstone is the official

Florida State Gem.

Moonstones are not

found in Florida . . .

They are also not

found on the moon !

Clearwater has more lightening strikes

per-capita than any city in the U.S.

The Alligator is the official

Florida State Reptile.

Gatorade was named for the

University of Florida Gators

where the drink was first developed.

The first mechanical refrigeration was

invented in Apalachicola in 1851.

FLORIDA LAWS

THE TOP 10 ACTS

FORBIDDEN BY LAW

IN THE STATE OF FLORIDA

THE TOP 10 ACTS

FORBIDDEN BY LAW

IN THE STATE OF FLORIDA

Showering naked

Singing in a public place

wearing a swimsuit

while in Sarasota

Harming a chicken in any way

while in Key West

Imitating an animal

while in Miami

Leaving a ship

while docked in Tampa Bay

if you are a rat

Parachuting on Sunday

if you are an

unmarried woman

Topless running or walking

within a 150 foot zone

between the beach and the street

Tying an elephant

to a parking meter

without paying the parking fee

Fishing from one’s car

while driving across a bridge

Having sexual relations

with a porcupine !

Shared Vision – Shared Purpose Shared Vision – Shared Purpose

Connecting with Other State Organizations

Strategies for Making Money

Current and Future Trends in Testing

NCTA RESOURCE LINKS

NCTA WEB SITE

Connecting with Other State Organizations

Strategies for Making Money

Take credit for your productivity increases

(doing more for the same money or with less).

Saving money is the same as making money.

Strategies For Making MoneyStrategies For Making Money

Team with your ConEd or Workforce

Development programs to provide

employment related assessment services.

Look for testing services you

provide for which you could

introduce or increase a fee.

Use the NCTA Mailing List to

locate opportunities for vendor

testing, with FACTA serving

as the initial point of contact.

Look for testing services you

provide for which you could

introduce or increase a fee.

Use the NCTA Mailing List to

locate opportunities for vender

testing, with FACTA serving

as the initial point of contact.

Take credit for your productivity increases

(doing more for the same money or with less).

Saving money is the same as making money.

Team with your ConEd or Workforce

Development programs to provide

employment related assessment services.

NCTA Grant ProgramNCTA Grant Program

Current and Future Trends in Testing

A very early lesson for

the future testing industry . . .

learned from the #2 pencil ! !

Slates and chalk continued to beused after the pencil was inventedin the early 19th Century due to the high costs for the manualproduction of pencils.

In 1866 a patent was grantedfor a wood-plane machineenabling mass production . . .

$ $ $

. . . the “new technology” (the pencil) was soon embraced in the marketplace.

Within 4 years over20 million pencilswere sold.

This enabled the unitcost of the pencil tobe lowered . . .

By 1903 the eraserwas addedand the cost dropped toa penny.

$ $ $

$ $ $

The testing industry has repeatedly confronted this same lesson . . .

$ $ $

$ $ $

Having the Technology is not enough.

Through mass

production and

robust distribution

channels, Manufacturing

has to provide the Market

the “right” unit cost !

Having what the

market needs is

not enough.

$ $ $

Lesson from the past – still relevant:

A BUSINESS MANDATE

FundingSources

ProductionCapability

MarketNeed

EnablingTechnology

DistributionChannels

ViableProductLow Unit

Price

ABusiness

Mandate

Universities and the CollegeBoard were operating their own commercial testing business

The Stanford Achievement Tests to assess subject knowledge

The Scholastic Aptitude Test operated by the College Board

The California Achievement Test used at the secondary level

The Iowa Test of Basic Skillsoperated at the University of Iowa

FundingSources

ProductionCapability

MarketNeed

EnablingTechnology

DistributionChannels

ViableProduct

Low UnitPrice

Increasing Demandfor Assessment

Private and Public Funding

Valid and Reliable Tests Manual Scoring and Reporting

In 1938 IBM creates the first electronicgraphite/optical test scoring machine.

Technology

Needed an Enabling Technology

$ $

$ $

FundingSources

ProductionCapability

MarketNeed

EnablingTechnology

DistributionChannels

ViableProduct

Low UnitPrice

Automated Scanning Scoring and Reporting

Measures performance relative to a

representative national sample of

students called a “norming sample”

Test Item DatabaseTest Item Database

Norm-Referenced Test

Test ItemStatistics

Test FormEquating

Test ItemRotation

Test Design

Validity &Reliability

% of Reusable Test Items

Reconstructed Test Forms

Large Test Volume

FundingSources

ProductionCapability

MarketNeed

EnablingTechnology

DistributionChannels

ViableProduct

Low UnitPrice

DistributionChannels

Labor forTest Delivery

InternationalPartners

Staff, Services and Facilities for Very Low Cost

FundingSources

ProductionCapability

MarketNeed

EnablingTechnology

DistributionChannels

ViableProduct

Low UnitPrice

National Norm Referenced

Paper & Pencil Testing

How did the industry get into this situation?

Is there a way out of it?

Service Issues

Delayed Reporting

Shipping Problems

Processing Delays

Delivery Problems

Scoring Errors

CBT significantly improvedthe test product . . . but it seriouslylimited production capabilities, required a new distribution infrastructure and raised operational and development costs

The delivery advantage of using schools(inexpensive classrooms for large group testing)

lost its business value for CBT.

CBT altered the business dynamics in the testing industry by reducing profit margins, expandingcompetition, and raising client expectationsbeyond what the vendors could profitably deliver.

Store-front commercial outlets: the new distribution channel.

-$

FundingSources

ProductionCapability

MarketNeed

EnablingTechnology

DistributionChannels

ViableProductLow Unit

Price

CBT disrupted the

dynamics of the P&P business model

CBT

http://www

Even internet delivery did not resolve things

Internet Internet Delivery Delivery

Additional competitors emerged

Cost savings were minimal

Test development and overall operating costs remained higher than traditional P&P

THEN CAME

NO CHILD SHALL BE LEFT BEHIND

WHICH MEANT

NO CHILD SHALL MOVE FORWARD

WITHOUT STANDARDIZED TESTING

At the outset, NCLB was seen by the

testing industry as a potential P&P

cash cow that could support the

marginal CBT side of the business

Low cost P&P delivery Less costly test development High test volumes Economies of scale

What turned the NCLB golden goose

into a turkey for the testing industry?

Individual State specs

Lower per test volumes

State owned test items

Short reporting windows

Delivery by the client

Low economies of scale

Increased competition

The Testing

Industry needs

a new Enabling

Technology

Measure performance relative to a

representative national sample of

students called a “norming sample”

Norm-Referenced Tests

Criterion-Referenced Tests

Measure performance relative

to mastery of content at certain

pre-defined levels of proficiency

NCLB Brought:

Mass Production Capability

Distribution Channels

Low Unit Cost

Complete the

“Business Mandate”

STILL NEEDS:

Future

Hand Held

Mini-PC Designs

Test Delivery Unit #4

Laptop Server + IDD’s

Test Delivery Unit #3

Laptop Server + IDD’s

Test Delivery Unit #2

Laptop Server + IDD’s

Test Delivery Unit #1

Laptop Server + MPC’s

Laptop Server Hub

Host Server

Laptop #1

Laptop #2

Laptop #3

Laptop #4

Test Server #1 (laptop)

Testing Room #4

Test Server #4 (laptop)

Testing Room #3

Test Server #1 (laptop)

Testing Room #2

Test Server #1 (laptop)

Testing Room #1

Test Server #1 (laptop)

A Future Model Data Source (Vendor)

Host and Laptop Serversstored in “wired” cabinet

Laptop Servers & hand heldMini-PCs transported inmobile Test Delivery Units

After testing in classroomsLaptop Servers returned to Host for test data upload.

Test data download throughSecure Internet Technology

No Costly PC Desktops NeededNo Costly PC Desktops Needed

No Internet Needed for DeliveryNo Internet Needed for Delivery

No Costly Test Servers NeededNo Costly Test Servers Needed

Large Capacity – Low Cost – High QualityLarge Capacity – Low Cost – High Quality

Test Delivery Business Paradigm ShiftTest Delivery Business Paradigm Shift

The Old Model in a New DimensionThe Old Model in a New Dimension

Returning the Test DeliveryReturning the Test Delivery

Business AdvantageBusiness Advantage

to the Schoolsto the Schools

FundingSources

MassProduction

MarketNeed

EnablingTechnology

DistributionChannels

ViableProduct

Low UnitPrice

Meeting The Business

Mandate

What other future technologies

may impact our testing profession?

What other future technologies

may impact our testing profession?

Confirming Examinee Identity Confirming Examinee Identity

Continuously During Testing Continuously During Testing

Confirming Examinee Identity Confirming Examinee Identity

Continuously During Testing Continuously During Testing

ComputerizedComputerized PaperPaper

Immediate Immediate

Answer SheetAnswer Sheet

andand

Essay ScoringEssay Scoring

ComputerizedComputerized

“ “Test Coat” Test Coat”

for electronic for electronic

test monitoringtest monitoring

Projection of a laser generated3-D image suspended in space

A future Test Center

with individualized virtual

testing environments and

projected Holographic images

as interactive test stimuli

In the future we may be able to

reach total standardization through

the management of each examinee’s

perceptions during the testing event.

In the future we may be able to

reach total standardization through

the management of each examinee’s

perceptions during the testing event.

Will future advances in brain mapping

and Psychopharmacology enable

selected modifications of the brain’s

perceptions to achieve absolute virtual

standardization - from the inside out?

Will future advances in brain mapping

and Psychopharmacology enable

selected modifications of the brain’s

perceptions to achieve absolute virtual

standardization - from the inside out?

Virtual Testing EnvironmentVirtual Testing EnvironmentVirtual Testing EnvironmentVirtual Testing Environment

Transmitting an Ideal, Standardized,

Virtual, Personal Testing Space,

Integrated within the Test

StandardizationStandardization

as an as an

Altered State ofAltered State of

Consciousness !Consciousness !

“It’s always better with a friend !”

“It’s always better with a friend !”

Dr. Louis Woodruff

NCTA President

Director of Testing

Bucks County Community College

Newtown, PA

woodruff@bucks.edu

215-968-8467