from captchas to captchæckers: can we automate usability and security evaluation of captchas?
TRANSCRIPT
![Page 1: From Captchas to Captchæckers: Can we automate usability and security evaluation of CAPTCHAs?](https://reader031.vdocuments.site/reader031/viewer/2022021816/58a9b0581a28ab9c758b62fb/html5/thumbnails/1.jpg)
From Captchas to Captchæckers:
Can we automate usability and
security evaluation of CAPTCHAs?
Shujun LI (李树钧)
Department of Computing
University of Surrey
http://www.hooklee.com
![Page 2: From Captchas to Captchæckers: Can we automate usability and security evaluation of CAPTCHAs?](https://reader031.vdocuments.site/reader031/viewer/2022021816/58a9b0581a28ab9c758b62fb/html5/thumbnails/2.jpg)
2
Starter 1: SONY CAPTCHA
- CAPTCHA @ SONY web forum (2011)
- In Google Chrome 21.0.1180.75 m:
- In Mozilla Firefox 15.0.1:
- In MSIE 9.0.8.112.16421:
- It is obviously weak, but…
![Page 3: From Captchas to Captchæckers: Can we automate usability and security evaluation of CAPTCHAs?](https://reader031.vdocuments.site/reader031/viewer/2022021816/58a9b0581a28ab9c758b62fb/html5/thumbnails/3.jpg)
3
Starter 2: an e-banking CAPTCHA
- CAPTCHA @ a Chinese bank’s e-banking login
Web page
- In all web browsers:
- It seems to be better than the previous one, but is not
really strong. However, the simplest way of breaking it is
… 5555555555555555555555555555555555555555555455555555555555555555555555555555555555555555555555555555
5555555555555555555555555555555555555551555545555555455555555555555555555555555555555555555555555555
5555555555555555555555555555555555555511555555555554555555555555555555555555555555555555555555555555
5555555555555555555555555555555555555115555555555555555555555555555555555000000005555555555555555555
5555545555555555555555555555554155555115555555555555333555555555555555500000000005555555555555555555
5555544222225555555555555555511445551155555555555555333333555555555555505555550005555555555555555555
5555542222222255555555555555551114551155555555555555333333335555555555555555500055555555555555555555
5555522255552255555555555555551111511555555555555553335555333555555555555555500055555555555555555555
5555522255552225555555555455555111511555555555555553335555333555555555555555000555555555555555555555
5555522255455222555555555545555111115555555555555553335555333555555555555555000455555555555555555555
5555522254445222555555555554555511115555555555555553333333333555555555555550005555555555555555555555
5555552225555222555555555555455511155555555555555553333333355555555555555550005555555555555555555555
5555552222552222555555555555555551155555555555555553335533355555555555555500005555555554555555555555
5555555522222222555555555555555551155555555555555553335553335555555555555500055555555445555555555555
5555555555555222555555555555555511155555555555555553335555333555555555555000055555554555555555555555
5555555555555222555555555555555511155555555555555553335555333555555555555000555500055555555555555555
5555555555555222555555555444455511155555555555555533335553333555555555544000000000055555555555555555
5555555254452225555555555555555511155555555555555553333333333555555555555440005555555555555555555555
5555555222222225555555555555555515555555555555555555555333335555555555555555555555555555555555555555
5555555552222555555555555555555555555555555555555555555555555555555555555555555555555555555555555555
5555555555555555555555555555555555555555555555555555555555555555555555555555555555555555555555555555
5455555555555555555555555555555555555555544455555555555555555555555555555555555555555555555555555555
5545555555555555555555555555555555555555555555555555555555555555555555555555555555555555555555455555
5545555555555555555555555555555555555555555555555555555555555555555555555555555555555555555555455555
5555555555555555555555555555555555555555555555555555555555555555555555555555555555555555555555545555
5555555555555555555555555555555555555555555555555555555555555555555555555555555555555555555555545555
5555555555555555555555555555555555555555555555555555555555555555555555555555555555555555555555554555
5555555555555555555555555555555555555555555555555555555555555555555555555555555555555555555555555555
![Page 4: From Captchas to Captchæckers: Can we automate usability and security evaluation of CAPTCHAs?](https://reader031.vdocuments.site/reader031/viewer/2022021816/58a9b0581a28ab9c758b62fb/html5/thumbnails/4.jpg)
4
Starter 3: CAPTCHA @ a Chinese site
- “Input the result of executing the above code
________ refresh the page to get other code”.
![Page 5: From Captchas to Captchæckers: Can we automate usability and security evaluation of CAPTCHAs?](https://reader031.vdocuments.site/reader031/viewer/2022021816/58a9b0581a28ab9c758b62fb/html5/thumbnails/5.jpg)
5
More starters: top 10 worst CAPTCHAs
- No. 1:
- No. 2:
- No. 3:
![Page 6: From Captchas to Captchæckers: Can we automate usability and security evaluation of CAPTCHAs?](https://reader031.vdocuments.site/reader031/viewer/2022021816/58a9b0581a28ab9c758b62fb/html5/thumbnails/6.jpg)
6
The main dishes
- Captchas (or CAPTCHAs)
- Security-Usability Dilemma
- Insecure but more usable CAPTCHAs
- Secure (?) but less usable CAPTCHAs
- CAPTCHA security and usability evaluation: what is the
current practice?
- Captchæckers (ongoing research)
- Automating usability evaluation
- Automating security evaluation
- Automating data collection
- Questions and Answers
![Page 7: From Captchas to Captchæckers: Can we automate usability and security evaluation of CAPTCHAs?](https://reader031.vdocuments.site/reader031/viewer/2022021816/58a9b0581a28ab9c758b62fb/html5/thumbnails/7.jpg)
Captchas Captchæckers
Captchas (or CAPTCHAs)
![Page 8: From Captchas to Captchæckers: Can we automate usability and security evaluation of CAPTCHAs?](https://reader031.vdocuments.site/reader031/viewer/2022021816/58a9b0581a28ab9c758b62fb/html5/thumbnails/8.jpg)
8
What are Captchas (or CAPTCHAs)?
- CAPTCHA
- Completely Automated Public Turing test to tell
Computers and Humans Apart
- It was proposed to fight against automated programs
abusing web resources (e.g. spamming).
I am human!
Then solve this!
![Page 9: From Captchas to Captchæckers: Can we automate usability and security evaluation of CAPTCHAs?](https://reader031.vdocuments.site/reader031/viewer/2022021816/58a9b0581a28ab9c758b62fb/html5/thumbnails/9.jpg)
9
CAPTCHA has many names!
- CAPTCHA: A Turing test?
- Automated Turing Test? – The human interrogator in a
Turing test is automated by a computer.
- Reversed Turing Test? – The role of something (human
interrogator) is reversed in a Turing test.
- CAPTCHA = HIP (Human Interactive Proof)?
- Historically, Blum et al. coined the term HIP to cover
many human-involved security systems including
CAPTCHA and HumanOID.
- So, CAPTCHA HIP.
- CAPTCHA = Authentication code?
- …
![Page 10: From Captchas to Captchæckers: Can we automate usability and security evaluation of CAPTCHAs?](https://reader031.vdocuments.site/reader031/viewer/2022021816/58a9b0581a28ab9c758b62fb/html5/thumbnails/10.jpg)
10
CAPTCHA: before the term was coined
- Moni Naor, Verification of a human in the
loop or identification via the Turing test, 1996
- , “Add-URL” web page, protected by
a scheme later known as CAPTCHA, 1997
- US Patent 6195698, Method for
selectively restricting access to computer
systems, filed on 13 April, 1998, issued on 27
February, 2001
- Jun Xu, Richard Lipton and Irfan Essa, Hello,
Are You Human? Georgia Institute of
Technology College of Computing Technical
Report, GIT-CC-00-28, 13 November 2000
![Page 11: From Captchas to Captchæckers: Can we automate usability and security evaluation of CAPTCHAs?](https://reader031.vdocuments.site/reader031/viewer/2022021816/58a9b0581a28ab9c758b62fb/html5/thumbnails/11.jpg)
11
CAPTCHA: after the term was coined
- 2000: Udi Manber from described the
“chat room problem” to Manuel Blum at the UC
Berkeley (who later moved to the CMU).
- 2000-2003: Blum and his collaborators coined the
term “CAPTCHA” and proposed some early
designs at www.captcha.net.
- 2002: the first report on
breaking CAPTCHAs appeared.
- 2002 onwards: a new kind of
cat-and-mouse game…
![Page 12: From Captchas to Captchæckers: Can we automate usability and security evaluation of CAPTCHAs?](https://reader031.vdocuments.site/reader031/viewer/2022021816/58a9b0581a28ab9c758b62fb/html5/thumbnails/12.jpg)
12
CAPTCHAs everywhere
- Many (most?) user registration web pages are
protected by CAPTCHAs.
- Many login pages and web forms as well.
![Page 13: From Captchas to Captchæckers: Can we automate usability and security evaluation of CAPTCHAs?](https://reader031.vdocuments.site/reader031/viewer/2022021816/58a9b0581a28ab9c758b62fb/html5/thumbnails/13.jpg)
13
CAPTCHAs everywhere
- CAPTCHA (reCAPTCHA) has been used for
digitizing books by Google.
![Page 14: From Captchas to Captchæckers: Can we automate usability and security evaluation of CAPTCHAs?](https://reader031.vdocuments.site/reader031/viewer/2022021816/58a9b0581a28ab9c758b62fb/html5/thumbnails/14.jpg)
14
CAPTCHAs everywhere
- CAPTCHA has been used as a new advertising
platform as well!
![Page 15: From Captchas to Captchæckers: Can we automate usability and security evaluation of CAPTCHAs?](https://reader031.vdocuments.site/reader031/viewer/2022021816/58a9b0581a28ab9c758b62fb/html5/thumbnails/15.jpg)
15
CAPTCHAs everywhere
- SweetCaptcha
- PlayThru
- MotionCAPTCHA
![Page 16: From Captchas to Captchæckers: Can we automate usability and security evaluation of CAPTCHAs?](https://reader031.vdocuments.site/reader031/viewer/2022021816/58a9b0581a28ab9c758b62fb/html5/thumbnails/16.jpg)
Captchas Captchæckers
Security-Usability Dilemma
![Page 17: From Captchas to Captchæckers: Can we automate usability and security evaluation of CAPTCHAs?](https://reader031.vdocuments.site/reader031/viewer/2022021816/58a9b0581a28ab9c758b62fb/html5/thumbnails/17.jpg)
17
Security-usability dilemma
- Let’s look at our good friend: (textual) passwords.
- Dinei Florêncio and Cormac Herley, A Large-Scale
Study of Web Password Habits, in Proc. WWW 2007,
ACM/W3C
![Page 18: From Captchas to Captchæckers: Can we automate usability and security evaluation of CAPTCHAs?](https://reader031.vdocuments.site/reader031/viewer/2022021816/58a9b0581a28ab9c758b62fb/html5/thumbnails/18.jpg)
18
Security-usability dilemma
- Let’s look at our good friend: (textual) passwords.
- DataGenetics, PIN analysis, 3 September 2012
xy00
xy9900xy 19xy
mmdd
xyxy
![Page 19: From Captchas to Captchæckers: Can we automate usability and security evaluation of CAPTCHAs?](https://reader031.vdocuments.site/reader031/viewer/2022021816/58a9b0581a28ab9c758b62fb/html5/thumbnails/19.jpg)
19
Security-usability dilemma
- So for passwords the dilemma is
- If a password is very strong (secure), then it
is not usable (hard to remember).
- If a password is usable (easy to remember),
then it is very weak (insecure).
- If I have to use a strong password but cannot
remember it, I will write it down!
- There is a similar one for CAPCTAHs!
- If a CAPTCHA is strong (hard for machines),
then it is hard to solve by humans.
- If a CAPTCHA is easy for humans to solve, it
is often weak (i.e., easy for machine as well).
![Page 20: From Captchas to Captchæckers: Can we automate usability and security evaluation of CAPTCHAs?](https://reader031.vdocuments.site/reader031/viewer/2022021816/58a9b0581a28ab9c758b62fb/html5/thumbnails/20.jpg)
20
Insecure but usable CAPTCHAs
- Almost all (if not all) e-banking CAPTCHAs [S. Li
et al. ACSAC 2010]
![Page 21: From Captchas to Captchæckers: Can we automate usability and security evaluation of CAPTCHAs?](https://reader031.vdocuments.site/reader031/viewer/2022021816/58a9b0581a28ab9c758b62fb/html5/thumbnails/21.jpg)
21
Strong but less usable CAPTCHAs
- Google CAPTCHA (not reCAPTCHA)
- Simplest are not very hard to solve
- Averagely OK?
- Some are very hard (if not impossible) to solve
- Google has replaced this CAPTCHA by reCAPTCHA for
user registration, but still keep it for login (only after
three continuous login errors occur).
![Page 22: From Captchas to Captchæckers: Can we automate usability and security evaluation of CAPTCHAs?](https://reader031.vdocuments.site/reader031/viewer/2022021816/58a9b0581a28ab9c758b62fb/html5/thumbnails/22.jpg)
22
CAPTCHA security mixed with usability
- Attackers are recruiting humans to do the job!
![Page 23: From Captchas to Captchæckers: Can we automate usability and security evaluation of CAPTCHAs?](https://reader031.vdocuments.site/reader031/viewer/2022021816/58a9b0581a28ab9c758b62fb/html5/thumbnails/23.jpg)
23
CAPTCHA security mixed with usability
- Attackers also know how to recruit humans without
even paying them a penny (since 2007)!
![Page 24: From Captchas to Captchæckers: Can we automate usability and security evaluation of CAPTCHAs?](https://reader031.vdocuments.site/reader031/viewer/2022021816/58a9b0581a28ab9c758b62fb/html5/thumbnails/24.jpg)
24
CAPTCHA usability evaluation
- So far, usability of CAPTCHAs is evaluated by
recruiting real human users.
- Problems?
- Time-consuming
- Scale-cost dilemma
- Hard to reproduce?
- …
![Page 25: From Captchas to Captchæckers: Can we automate usability and security evaluation of CAPTCHAs?](https://reader031.vdocuments.site/reader031/viewer/2022021816/58a9b0581a28ab9c758b62fb/html5/thumbnails/25.jpg)
25
CAPTCHA security evaluation
- Security evaluation = Attack discovery
- Is security evaluation easy?
- Yes: if a CAPTCHA is weak, it is easy.
- No: if a CAPTCHA is strong, it can be very difficult.
- It depends on the security analyst’s experience!
- Problems?
- Slight change of CAPTCHA design may require big
changes to source code of the attack.
- Even if changes are not heavy nor difficult, it’s boring
and error-prone to do them all by hand!
- Porting from one programming language to the other
can be difficult.
![Page 26: From Captchas to Captchæckers: Can we automate usability and security evaluation of CAPTCHAs?](https://reader031.vdocuments.site/reader031/viewer/2022021816/58a9b0581a28ab9c758b62fb/html5/thumbnails/26.jpg)
26
CAPTCHA security evaluation
- Remarks of a reCAPTCHA cracker [Chad W.
Houck, DEFCON’2010]
- “Unfortunately any CAPTCHA that can be read by a
human can eventually be read by a computer. The only
solution is to stay one step ahead of those wishing to
abuse these systems by consistently changing the
CAPTCHA distortions and design. While it may take
the maintainer of a CAPTCHA system a couple of
hours to implement a change, it takes a human no
time to adjust to the difference, while a person
wishing to keep their automated system working
that defeats the CAPTCHA may take weeks to adopt
the changes necessary to get it running again.”
![Page 27: From Captchas to Captchæckers: Can we automate usability and security evaluation of CAPTCHAs?](https://reader031.vdocuments.site/reader031/viewer/2022021816/58a9b0581a28ab9c758b62fb/html5/thumbnails/27.jpg)
27
Multi-CAPTCHA engines
- Some web sites (or CAPTCHA service providers)
have started deploying multi-CAPTCHA engines.
- The idea is simple:
- include a (large) number of different CAPTCHA
schemes in the engine;
- (randomly) select a scheme to generate each new
CAPTCHA;
- the CAPTCHA poll and selection rule may be
customized by the user.
- The consequence
- Security and usability evaluation complexity
![Page 28: From Captchas to Captchæckers: Can we automate usability and security evaluation of CAPTCHAs?](https://reader031.vdocuments.site/reader031/viewer/2022021816/58a9b0581a28ab9c758b62fb/html5/thumbnails/28.jpg)
28
Multi-CAPTCHA engines
- One example
![Page 29: From Captchas to Captchæckers: Can we automate usability and security evaluation of CAPTCHAs?](https://reader031.vdocuments.site/reader031/viewer/2022021816/58a9b0581a28ab9c758b62fb/html5/thumbnails/29.jpg)
29
Multi-CAPTCHA engines
- Another example: Microsoft live.com CAPTCHAs?
- An insider told me a powerful multi-CAPTCHA engine
was (is?) in place which is able to produce many
different types of CAPTCHAs.
- I didn’t observe a diversity of CAPTCHA on live.com
user registration Web page.
(reCAPTCHA, 2010, 2011-2012?)
(2011, 2012?)
(2012, also in 2010-2011?)
- The insider explained the Microsoft staff who was
managing the engine has trouble (re-)reconfiguring it!
![Page 30: From Captchas to Captchæckers: Can we automate usability and security evaluation of CAPTCHAs?](https://reader031.vdocuments.site/reader031/viewer/2022021816/58a9b0581a28ab9c758b62fb/html5/thumbnails/30.jpg)
30
Security vs. usability
- A balance between security and usability
- No usability evaluation, no security evaluation
- (Automated) usability+security evaluation
- effort a legitimate user needs to solve a CAPTCHA
- costs of human solver based attacks
- (automatic) usability control of CAPCTHAs
- (automatic) reconfiguration of CAPTCHAs
- …
![Page 31: From Captchas to Captchæckers: Can we automate usability and security evaluation of CAPTCHAs?](https://reader031.vdocuments.site/reader031/viewer/2022021816/58a9b0581a28ab9c758b62fb/html5/thumbnails/31.jpg)
31
Now come the research questions…
- Can we automate the security and usability
evaluation of CAPTCHAs?
- If full automation is not possible, can we automate
part of the process?
- What techniques do we need?
- How do we validate results of automated
evaluation?
- How can we link automated security and usability
evaluation?
- …
![Page 32: From Captchas to Captchæckers: Can we automate usability and security evaluation of CAPTCHAs?](https://reader031.vdocuments.site/reader031/viewer/2022021816/58a9b0581a28ab9c758b62fb/html5/thumbnails/32.jpg)
Captchas Captchæckers
Captchæckers: The solution?
(Ongoing/Incomplete research)
![Page 33: From Captchas to Captchæckers: Can we automate usability and security evaluation of CAPTCHAs?](https://reader031.vdocuments.site/reader031/viewer/2022021816/58a9b0581a28ab9c758b62fb/html5/thumbnails/33.jpg)
33
What is a Captchæcker?
- Captchæcker = Captcha + checker
- It is a term created by my collaborators and me in
2011.
- A Captchæcker = A fully or partly automated
programme that can evaluate one or more
performance aspects of CAPTCHAs
- We consider security and usability only in this research.
- Captchæckers a benchmarking toolbox for
CAPTCHA performance evaluation
- Captchæckers a CAPTCHA reconfiguration
toolbox as well!
![Page 34: From Captchas to Captchæckers: Can we automate usability and security evaluation of CAPTCHAs?](https://reader031.vdocuments.site/reader031/viewer/2022021816/58a9b0581a28ab9c758b62fb/html5/thumbnails/34.jpg)
34
Usability Captchæcker
- The input: one CAPTCHA
- The output: usability metric(s) of the input
- Hardness as the first impression perceived by an
average human user: subjective
- Hardness as an experience perceived by an average
human user after trying to solve it: subjective
- Hardness measured as the average response time and
error rate of the target user population: objective
- The response time may be misleading if a user gives
up earlier (so response time is related to error rate)
- The key research question: is there a predictable
pattern in the average behavior of human users?
![Page 35: From Captchas to Captchæckers: Can we automate usability and security evaluation of CAPTCHAs?](https://reader031.vdocuments.site/reader031/viewer/2022021816/58a9b0581a28ab9c758b62fb/html5/thumbnails/35.jpg)
35
Usability Captchæcker: preliminary work
- Our positive answer via a small-scale user study
[S. Li et al., SOUPS 2011 poster]
- An artificial neutral network predicted hardness of 38
CAPTCHAs perceived by 5 users with >80% accuracy.
- Only four simple geometric features are involved.
4 6 8 10 12 14 16 18 20-9
-8
-7
-6
-5
-4
-3
-2
-1
0CL and ET Values and User Rating
Compact-Length (CL)
Eu
ler-T
hic
kn
ess
(E
T)
Extremely Easy
Somewhat Easy
Somewhat Difficult
Difficult but Readable
Impossible to Read4 6 8 10 12 14 16 18 20
-5
-4.5
-4
-3.5
-3
-2.5
-2
-1.5
-1
-0.5
0CL and ET Values and User Rating
Compactness-Length (CL)
Eu
ler-
Th
ick
nes
s (E
T)
Extremely Easy
Somewhat Easy
Somewhat Difficult
Difficult but Readable
![Page 36: From Captchas to Captchæckers: Can we automate usability and security evaluation of CAPTCHAs?](https://reader031.vdocuments.site/reader031/viewer/2022021816/58a9b0581a28ab9c758b62fb/html5/thumbnails/36.jpg)
36
Usability Captchæcker: future work
- Add pre-processing steps (e.g. denoising)
- Increase the number of CAPTCHAs trained/tested
- Increase the number of human users involved and
the diversity of users (cultural background, age,
computer knowledge, etc.)
- Test more geometric and non-geometric features
- Try to predict all three editions of hardness
- Try more machine learning algorithms
- Hierarchical classification, ensemble methods, …
- …
![Page 37: From Captchas to Captchæckers: Can we automate usability and security evaluation of CAPTCHAs?](https://reader031.vdocuments.site/reader031/viewer/2022021816/58a9b0581a28ab9c758b62fb/html5/thumbnails/37.jpg)
37
Security Captchæcker
- The input: a number of CAPTCHAs generated by
the same scheme
- The output: security metric(s) of the input
- For each attack, the metric reports how strong the
CAPCHA scheme is against this specific attack
(percentage of fully recognized CAPTCHAs)
- The key questions: how to implement different
attacks more effectively and how to discover new
unknown attacks?
![Page 38: From Captchas to Captchæckers: Can we automate usability and security evaluation of CAPTCHAs?](https://reader031.vdocuments.site/reader031/viewer/2022021816/58a9b0581a28ab9c758b62fb/html5/thumbnails/38.jpg)
38
Security Captchæcker
- Some previous work showed attacks on many different
CAPTCHAs are based on the same set of techniques [S. Li
et al., ACSAC 2010]
- Another one: Bursztein et al., Text-based CAPTCHA
Strengths and Weaknesses, in Proc. ACM CCS 2011
Morphological
Operations
Line Detection
Image
Inpainting
Genuine
CAPTCHA images k-means Layer
Segmentation
Character
Segmentation
Character
Recognition
CAPTCHA Image
Synthesis
Image
Inpainting
Forged
CAPTCHA image
![Page 39: From Captchas to Captchæckers: Can we automate usability and security evaluation of CAPTCHAs?](https://reader031.vdocuments.site/reader031/viewer/2022021816/58a9b0581a28ab9c758b62fb/html5/thumbnails/39.jpg)
39
Security Captchæcker
- The full attack space is prohibitively huge.
- Full automation is (computationally) impossible.
- Partial automation should be possible, where the
human expert needs to define a space of attacks.
- The space of attacks may be developed using the
new (ISO/IEC) standardized dataflow
programming framework RVC (Reconfigurable
Video Coding, to be extended to RMC =
Reconfigurable Multimedia Coding).
![Page 40: From Captchas to Captchæckers: Can we automate usability and security evaluation of CAPTCHAs?](https://reader031.vdocuments.site/reader031/viewer/2022021816/58a9b0581a28ab9c758b62fb/html5/thumbnails/40.jpg)
40
What is RVC?
- A new ISO/IEC standard made by MEPG for
developing complicated multimedia codecs
Actions
State
Variables
Implementation-
Independent Design
C
Java
LLVM
VHDL
Verilog
C++
Target
Implementations
![Page 41: From Captchas to Captchæckers: Can we automate usability and security evaluation of CAPTCHAs?](https://reader031.vdocuments.site/reader031/viewer/2022021816/58a9b0581a28ab9c758b62fb/html5/thumbnails/41.jpg)
41
Automating data collection
- A database of CAPTCHAs
- How to select CAPTCHAs?
- How to extract CAPTCHAs from web pages?
- Automating the selection and extraction processes (A
Web crawler for CAPTCHAs) is very useful.
- A database of CAPTCHA usability data
- Collecting data from real human users is costly and time
consuming.
- Crowdsourcing can help?
- Automating the data collection process from
crowdsourcing Web sites will be very useful.
![Page 42: From Captchas to Captchæckers: Can we automate usability and security evaluation of CAPTCHAs?](https://reader031.vdocuments.site/reader031/viewer/2022021816/58a9b0581a28ab9c758b62fb/html5/thumbnails/42.jpg)
42
Captchæckers for whom?
- Researchers
- Who want to have a deeper understanding of different elements in
the system.
- End users
- Who want to know more about CAPTCHAs they’re using.
- Webmasters
- Who want to select a right CAPTCHA scheme for their Web sites.
- CAPTCHA service providers
- Who want to serve their customers better and improve their
products.
- CAPTCHA solving service providers
- Who want a more accurate estimate of the costs for better pricing.
- …
![Page 43: From Captchas to Captchæckers: Can we automate usability and security evaluation of CAPTCHAs?](https://reader031.vdocuments.site/reader031/viewer/2022021816/58a9b0581a28ab9c758b62fb/html5/thumbnails/43.jpg)
43
Captchæckers for end users
- Only evaluation
Security
Captchæcker
Usability
Captchæcker
Overall
metric(s)
Crowd-
sourcing
CAPTCHA
Usability
Database
CAPTCHA-
Breaking
Tool Library
CAPTCHA(s)
CAPTCHA
Database
CAPTCHA
Web Crawler
![Page 44: From Captchas to Captchæckers: Can we automate usability and security evaluation of CAPTCHAs?](https://reader031.vdocuments.site/reader031/viewer/2022021816/58a9b0581a28ab9c758b62fb/html5/thumbnails/44.jpg)
44
Captchæckers for Webmasters
- Evaluation + reconfiguration [S. Li et al.,
SafeConfig 2011]
PRNGCAPTCHA
Engine
Security
Captchæcker
Usability
Captchæcker
Reconfigurator
Crowd-
sourcing
CAPTCHA
Usability
Database
CAPTCHA-
Breaking
Tool Library
CAPTCHA
Database
Side Information
CAPTCHAs
Own Web
site(s)
CAPTCHA
Web Crawler
![Page 45: From Captchas to Captchæckers: Can we automate usability and security evaluation of CAPTCHAs?](https://reader031.vdocuments.site/reader031/viewer/2022021816/58a9b0581a28ab9c758b62fb/html5/thumbnails/45.jpg)
45
Can we go beyond CAPTCHAs?
- CAPTCHA is just one kind of computer security
system involving human users.
- Can we automate security and usability evaluation
of other human-involved computer security
systems? And if so how?
- Other HIPs? (HumanOID?)
- User authentication systems (e.g. graphical passwords)
- HCI of security software (firewall, anti-virus software,
etc.)
- Security warning systems (unprotected HTTP traffic,
saving passwords in cookie, suspicious web sites, etc.)
- …
![Page 46: From Captchas to Captchæckers: Can we automate usability and security evaluation of CAPTCHAs?](https://reader031.vdocuments.site/reader031/viewer/2022021816/58a9b0581a28ab9c758b62fb/html5/thumbnails/46.jpg)
Captchas Captchæckers
Thanks for your attention!
Questions + Answers
Collaborations?