1 examination accreditation program exam security: preventing exam espionage saturday, october 1...
TRANSCRIPT
![Page 1: 1 Examination Accreditation Program Exam Security: Preventing Exam Espionage Saturday, October 1 Baltimore – Maryland - USA](https://reader030.vdocuments.site/reader030/viewer/2022032313/56649e6b5503460f94b69e2f/html5/thumbnails/1.jpg)
1
Examination Accreditation Program
Exam Security:Preventing Exam Espionage
Saturday, October 1Baltimore – Maryland - USA
![Page 2: 1 Examination Accreditation Program Exam Security: Preventing Exam Espionage Saturday, October 1 Baltimore – Maryland - USA](https://reader030.vdocuments.site/reader030/viewer/2022032313/56649e6b5503460f94b69e2f/html5/thumbnails/2.jpg)
2
Exam Security: Introducing the Presenters
During Test Development
Lawrence J. Fabrey, Ph.D.
During Test AdministrationDave Moore
After the TestChris Beer
Tadas Dabsys
![Page 3: 1 Examination Accreditation Program Exam Security: Preventing Exam Espionage Saturday, October 1 Baltimore – Maryland - USA](https://reader030.vdocuments.site/reader030/viewer/2022032313/56649e6b5503460f94b69e2f/html5/thumbnails/3.jpg)
3
Benefits of ARELLO-Accreditation
Independent Evaluation Confidence and Protection Defensible Standards Reasonable Consistency in Quality Facilitates License Recognition
www.ARELLO.org (ARELLO Guidelines for Accreditation)
![Page 4: 1 Examination Accreditation Program Exam Security: Preventing Exam Espionage Saturday, October 1 Baltimore – Maryland - USA](https://reader030.vdocuments.site/reader030/viewer/2022032313/56649e6b5503460f94b69e2f/html5/thumbnails/4.jpg)
4
ARELLO Additional Resources
About Exam Development An Accreditation Overview ARELLO® Matters to Consider in Exam RFP ARELLO® Top Ten Questions and Complaints on
Licensing Exams ARELLO® Supreme Court Case - Exam Validity ARELLO® Examination Accreditation Program
- White Paper on Determining Minimum Passing Scores
![Page 5: 1 Examination Accreditation Program Exam Security: Preventing Exam Espionage Saturday, October 1 Baltimore – Maryland - USA](https://reader030.vdocuments.site/reader030/viewer/2022032313/56649e6b5503460f94b69e2f/html5/thumbnails/5.jpg)
5
Security During Examination Development
• Physical Environment • Electronic Access • Staffing • Inventory Storage• Subject Matter Experts• Other Functions
• Translation, Printing, Artwork, etc.
![Page 6: 1 Examination Accreditation Program Exam Security: Preventing Exam Espionage Saturday, October 1 Baltimore – Maryland - USA](https://reader030.vdocuments.site/reader030/viewer/2022032313/56649e6b5503460f94b69e2f/html5/thumbnails/6.jpg)
6
Development – Physical Access
• Secure Room/Area • Vault – Key Access • Limited Authorized Access• Chain-of-Custody• Shredding – including notes/drafts
![Page 7: 1 Examination Accreditation Program Exam Security: Preventing Exam Espionage Saturday, October 1 Baltimore – Maryland - USA](https://reader030.vdocuments.site/reader030/viewer/2022032313/56649e6b5503460f94b69e2f/html5/thumbnails/7.jpg)
7
Development – Electronic Access
• Passwords & Screensavers• Selective Access Permission • Specific Time Period• Read-only, non-print, non-export• Webinars
![Page 8: 1 Examination Accreditation Program Exam Security: Preventing Exam Espionage Saturday, October 1 Baltimore – Maryland - USA](https://reader030.vdocuments.site/reader030/viewer/2022032313/56649e6b5503460f94b69e2f/html5/thumbnails/8.jpg)
8
Development - Staffing
• Staff • Confidentiality Agreements • Conflict of Interest • Audits
![Page 9: 1 Examination Accreditation Program Exam Security: Preventing Exam Espionage Saturday, October 1 Baltimore – Maryland - USA](https://reader030.vdocuments.site/reader030/viewer/2022032313/56649e6b5503460f94b69e2f/html5/thumbnails/9.jpg)
9
Development - SME’s and Other Functions
• Subject Matter Experts • Translators• Artwork• Supplies for Practical Exams• Printers• Couriers
![Page 10: 1 Examination Accreditation Program Exam Security: Preventing Exam Espionage Saturday, October 1 Baltimore – Maryland - USA](https://reader030.vdocuments.site/reader030/viewer/2022032313/56649e6b5503460f94b69e2f/html5/thumbnails/10.jpg)
10
Development – Inventory and Distribution
• Printing (Serial #’s, Shrink wrap)• Shipping (Traceable, Armed Courier)• Electronic Transmission• Web Access
![Page 11: 1 Examination Accreditation Program Exam Security: Preventing Exam Espionage Saturday, October 1 Baltimore – Maryland - USA](https://reader030.vdocuments.site/reader030/viewer/2022032313/56649e6b5503460f94b69e2f/html5/thumbnails/11.jpg)
11
Security During Exam Administration
• Paper & Pencil • Computer-based Testing• Practical Exams• Open-book Exams• Special Accommodations • Facilities & Staff• Candidates
![Page 12: 1 Examination Accreditation Program Exam Security: Preventing Exam Espionage Saturday, October 1 Baltimore – Maryland - USA](https://reader030.vdocuments.site/reader030/viewer/2022032313/56649e6b5503460f94b69e2f/html5/thumbnails/12.jpg)
12
Administration – Paper & Pencil
• Printing • Shipping Receipt and Inventory• Storage• Contingency Plan• Return Shipment & Inventory
![Page 13: 1 Examination Accreditation Program Exam Security: Preventing Exam Espionage Saturday, October 1 Baltimore – Maryland - USA](https://reader030.vdocuments.site/reader030/viewer/2022032313/56649e6b5503460f94b69e2f/html5/thumbnails/13.jpg)
13
Administration - CBT
• CBT Centers • Restricted Login Access• Restricted Time Periods• Restricted Printing• Cameras, DVR’s
![Page 14: 1 Examination Accreditation Program Exam Security: Preventing Exam Espionage Saturday, October 1 Baltimore – Maryland - USA](https://reader030.vdocuments.site/reader030/viewer/2022032313/56649e6b5503460f94b69e2f/html5/thumbnails/14.jpg)
14
Administration – Special Administrations
• Practical Exams • Raters (SME) – Conflict of Interest• Supplies
![Page 15: 1 Examination Accreditation Program Exam Security: Preventing Exam Espionage Saturday, October 1 Baltimore – Maryland - USA](https://reader030.vdocuments.site/reader030/viewer/2022032313/56649e6b5503460f94b69e2f/html5/thumbnails/15.jpg)
15
Administration – Open-book
• Open-book Exams • Notes• Highlight/Underline• Post-its/Dividers• Screening
![Page 16: 1 Examination Accreditation Program Exam Security: Preventing Exam Espionage Saturday, October 1 Baltimore – Maryland - USA](https://reader030.vdocuments.site/reader030/viewer/2022032313/56649e6b5503460f94b69e2f/html5/thumbnails/16.jpg)
16
Administration - Facilities
• Limited Access• Spacing Candidates• Monitor Restroom Breaks• Secure Storage - Supplements• Special Accommodations
![Page 17: 1 Examination Accreditation Program Exam Security: Preventing Exam Espionage Saturday, October 1 Baltimore – Maryland - USA](https://reader030.vdocuments.site/reader030/viewer/2022032313/56649e6b5503460f94b69e2f/html5/thumbnails/17.jpg)
17
Administration - Staffing
• Conflict of Interest• Self-regulating• Restricted Access (Email, Internet)• Audits• Training – Active Proctoring• Termination
![Page 18: 1 Examination Accreditation Program Exam Security: Preventing Exam Espionage Saturday, October 1 Baltimore – Maryland - USA](https://reader030.vdocuments.site/reader030/viewer/2022032313/56649e6b5503460f94b69e2f/html5/thumbnails/18.jpg)
18
Administration - Special
• Special Accommodations• Interpreters
![Page 19: 1 Examination Accreditation Program Exam Security: Preventing Exam Espionage Saturday, October 1 Baltimore – Maryland - USA](https://reader030.vdocuments.site/reader030/viewer/2022032313/56649e6b5503460f94b69e2f/html5/thumbnails/19.jpg)
19
Administration - Candidates
• Candidates• Identification• Roster Signature• Fingerprint Login• Clothing/Purses• Activity/Mannerisms
![Page 20: 1 Examination Accreditation Program Exam Security: Preventing Exam Espionage Saturday, October 1 Baltimore – Maryland - USA](https://reader030.vdocuments.site/reader030/viewer/2022032313/56649e6b5503460f94b69e2f/html5/thumbnails/20.jpg)
20
Available through SkyMall and Web
![Page 21: 1 Examination Accreditation Program Exam Security: Preventing Exam Espionage Saturday, October 1 Baltimore – Maryland - USA](https://reader030.vdocuments.site/reader030/viewer/2022032313/56649e6b5503460f94b69e2f/html5/thumbnails/21.jpg)
21
Caught on Camera
Smart phone
Notes
![Page 22: 1 Examination Accreditation Program Exam Security: Preventing Exam Espionage Saturday, October 1 Baltimore – Maryland - USA](https://reader030.vdocuments.site/reader030/viewer/2022032313/56649e6b5503460f94b69e2f/html5/thumbnails/22.jpg)
22
I know I will be Caught
![Page 23: 1 Examination Accreditation Program Exam Security: Preventing Exam Espionage Saturday, October 1 Baltimore – Maryland - USA](https://reader030.vdocuments.site/reader030/viewer/2022032313/56649e6b5503460f94b69e2f/html5/thumbnails/23.jpg)
23
![Page 24: 1 Examination Accreditation Program Exam Security: Preventing Exam Espionage Saturday, October 1 Baltimore – Maryland - USA](https://reader030.vdocuments.site/reader030/viewer/2022032313/56649e6b5503460f94b69e2f/html5/thumbnails/24.jpg)
24
Show Me the Money!
![Page 25: 1 Examination Accreditation Program Exam Security: Preventing Exam Espionage Saturday, October 1 Baltimore – Maryland - USA](https://reader030.vdocuments.site/reader030/viewer/2022032313/56649e6b5503460f94b69e2f/html5/thumbnails/25.jpg)
25
Security After the Test
• Data Forensics• Monitoring Websites• Item Updating• Facility Screening• Audits• Incident Reporting
![Page 26: 1 Examination Accreditation Program Exam Security: Preventing Exam Espionage Saturday, October 1 Baltimore – Maryland - USA](https://reader030.vdocuments.site/reader030/viewer/2022032313/56649e6b5503460f94b69e2f/html5/thumbnails/26.jpg)
26
After – Data Forensics - Items
p-value: Percentage of candidates who answer an item correctly
pbis: Correlation between performance on an item and performance on the overall exam
option stats: Percentage of candidates who answer each option correctly
option pbis: Correlation between performance on the options and the overall exam
![Page 27: 1 Examination Accreditation Program Exam Security: Preventing Exam Espionage Saturday, October 1 Baltimore – Maryland - USA](https://reader030.vdocuments.site/reader030/viewer/2022032313/56649e6b5503460f94b69e2f/html5/thumbnails/27.jpg)
27
After – Data Forensics - Items
p-value: Too high, too low
pbis: Too low
options:Distractors with no responses
Distractors with more responses than key
O pbis: Distractors higher than key
Omits Above expectations
![Page 28: 1 Examination Accreditation Program Exam Security: Preventing Exam Espionage Saturday, October 1 Baltimore – Maryland - USA](https://reader030.vdocuments.site/reader030/viewer/2022032313/56649e6b5503460f94b69e2f/html5/thumbnails/28.jpg)
28
After – Data Forensics - Candidates
Suspicious Candidate Activity
Test for significant differences in pass rates Better performance on harder items High scores in low test times Suspicious improvement in scores
![Page 29: 1 Examination Accreditation Program Exam Security: Preventing Exam Espionage Saturday, October 1 Baltimore – Maryland - USA](https://reader030.vdocuments.site/reader030/viewer/2022032313/56649e6b5503460f94b69e2f/html5/thumbnails/29.jpg)
29
After – Monitoring Websites
• Cram Schools• Sample Exams• False Advertising• Social Networking
![Page 30: 1 Examination Accreditation Program Exam Security: Preventing Exam Espionage Saturday, October 1 Baltimore – Maryland - USA](https://reader030.vdocuments.site/reader030/viewer/2022032313/56649e6b5503460f94b69e2f/html5/thumbnails/30.jpg)
30
After – Item Updating
• Monitor Item Statistics• Replace Items• Add New Items
![Page 31: 1 Examination Accreditation Program Exam Security: Preventing Exam Espionage Saturday, October 1 Baltimore – Maryland - USA](https://reader030.vdocuments.site/reader030/viewer/2022032313/56649e6b5503460f94b69e2f/html5/thumbnails/31.jpg)
31
After – Facility Screening
• Look for Notes in Testing Station• Screen Restrooms and Hallways
![Page 32: 1 Examination Accreditation Program Exam Security: Preventing Exam Espionage Saturday, October 1 Baltimore – Maryland - USA](https://reader030.vdocuments.site/reader030/viewer/2022032313/56649e6b5503460f94b69e2f/html5/thumbnails/32.jpg)
32
After - Audits
• Test Center Audits• “Secret Shopper” Programs• Inventory• Continuing Education for Proctors
![Page 33: 1 Examination Accreditation Program Exam Security: Preventing Exam Espionage Saturday, October 1 Baltimore – Maryland - USA](https://reader030.vdocuments.site/reader030/viewer/2022032313/56649e6b5503460f94b69e2f/html5/thumbnails/33.jpg)
33
After – Incident Reporting
• Documentation• Evidence• Reporting to Client
![Page 34: 1 Examination Accreditation Program Exam Security: Preventing Exam Espionage Saturday, October 1 Baltimore – Maryland - USA](https://reader030.vdocuments.site/reader030/viewer/2022032313/56649e6b5503460f94b69e2f/html5/thumbnails/34.jpg)
34
Examples
• Cheating Incidents
![Page 35: 1 Examination Accreditation Program Exam Security: Preventing Exam Espionage Saturday, October 1 Baltimore – Maryland - USA](https://reader030.vdocuments.site/reader030/viewer/2022032313/56649e6b5503460f94b69e2f/html5/thumbnails/35.jpg)
35
ARELLO Accredited Providers
Applied Measurement Professionalshttp://www.goamp.com
Pearson VUEhttp://www.pearsonvue.com
PSI Services LLChttp://psionline.com
![Page 36: 1 Examination Accreditation Program Exam Security: Preventing Exam Espionage Saturday, October 1 Baltimore – Maryland - USA](https://reader030.vdocuments.site/reader030/viewer/2022032313/56649e6b5503460f94b69e2f/html5/thumbnails/36.jpg)
36
QUESTIONS?COMMENTS!
Fabulous Prize Giveaway