migrating your paper-n-pencil test to the computer with security in mind jamie mulkey caveon test...
TRANSCRIPT
Migrating your Paper-n-Pencil Test to the
Computer with Security in Mind
Jamie Mulkey Caveon Test Security
Laura Culver-EdgarNCCAOM
® Caveon, 2006
Secure Test Administration
RemediationPrevention Detection
® Caveon, 2006
Agenda
Why move to CBT? Prevention (Before)
Policy and process considerations Detection (During)
Monitoring candidates Remediation (After)
Candidate, record, and data management
® Caveon, 2006
Making the decision to move to CBT Better access and distribution of
testing program More efficient testing More flexible testing Ability to develop/deliver
innovative item types Better test security…
® Caveon, 2006
Security Advantages
Ability to turn items on and off as needed. Bad items, compromised items, overexposed
items. Unlikely that people will get the same
items CBT, especially CAT can help in this regard
Ability to investigate item-level data better Response time, unexpected change in ability
during the test, etc. Faster reporting mechanisms
Important in cases of irregular behavior
® Caveon, 2006
Test Administration (Before)
Proper identification procedures Define authorized/unauthorized
devices Process to report any discrepancies Proctors and other test
administration personnel trained Interventions are used when
suspicious activity occurs
® Caveon, 2006
Security Measures - Biometrics
® Caveon, 2006
Testing continuously vs. testing windows Size of testing population and the availability
of testing sites/seats Length of test and availability of testing
sites/seats Monitoring of item exposure through internet
braindumping/forums/chatrooms Monitoring of possible item exposure in
various time zones Results to candidate are released
immediately or after a period of time Size of your item pool Test administration: form-based or CAT
® Caveon, 2006
Test Administration (During)
Managing candidate identification biometrics
Continuous monitoring or examinees Personnel, video/audio monitoring
Physical control over unauthorized material and devices Physical space to hold personal items
Taking advantage of computer technology
Managing an intervention when suspicious activity occurs Halt testing, notification and escalation
® Caveon, 2006
Confirming exam exposure with Data Forensics™Helps validate level of
exposure Has a test been compromised
and to what extent? When and where was it
compromised? Who are the potential
culprits?
® Caveon, 2006
Caveon Case Study: Global Certification Exams - Effect on Test Performance
® Caveon, 2006Incidence of Test Abuse by Exam
Exam 425 has experienced the greatest incidence of test abuse with suspected test abuse occurring in ~180 of every 1000 tests.
High instance of collusion and cheating indicate probable sale and/or sharing of test content on the Internet.
® Caveon, 2006
Test Administration (After)
Relevant data immediately forwarded to the test sponsor
Score validation and reporting Using Data Forensics to Identify
and: Invalidate scores Manage rogue test centers Manage item exposure
® Caveon, 2006
Remediation
Execution of policies Incident response matrix
Sanctions for poor-performing sites Investigation Warning Closure
Cease & Desist / Take down
® Caveon, 2006
Tier 1
® Caveon, 2006
Security Planning
Roles and Responsibilities Security Budget Policies and Procedures Putting an overall plan together
® Caveon, 2006
Questions & Answers
® Caveon, 2006
Thanks for attending!
Jamie Mulkey, [email protected]
916 652-4017 phonewww.caveon.com
Laura Culver-Edgar, MBA, MT(ASCP) 703 299-5891
Please contact us: