fuzzing the rust typechecker using clp - github pages · fuzzing the rust typechecker using clp...
TRANSCRIPT
![Page 1: Fuzzing the Rust Typechecker Using CLP - GitHub Pages · Fuzzing the Rust Typechecker Using CLP Kyle Dewey, Jared Roesch, Ben Hardekopf University of California, Santa Barbara. Teaser](https://reader030.vdocuments.site/reader030/viewer/2022040410/5ecccf5ec221095fc21e2b58/html5/thumbnails/1.jpg)
Fuzzing the Rust Typechecker Using CLP
Kyle Dewey, Jared Roesch, Ben Hardekopf
University of California, Santa Barbara
![Page 2: Fuzzing the Rust Typechecker Using CLP - GitHub Pages · Fuzzing the Rust Typechecker Using CLP Kyle Dewey, Jared Roesch, Ben Hardekopf University of California, Santa Barbara. Teaser](https://reader030.vdocuments.site/reader030/viewer/2022040410/5ecccf5ec221095fc21e2b58/html5/thumbnails/2.jpg)
Teaser
• We identify three kinds of bugs that typecheckers can exhibit
• We describe general techniques for automatically finding these kinds of bugs
• We apply these ideas to testing the Rust programming language, and find 14 developer-confirmed bugs
2
![Page 3: Fuzzing the Rust Typechecker Using CLP - GitHub Pages · Fuzzing the Rust Typechecker Using CLP Kyle Dewey, Jared Roesch, Ben Hardekopf University of California, Santa Barbara. Teaser](https://reader030.vdocuments.site/reader030/viewer/2022040410/5ecccf5ec221095fc21e2b58/html5/thumbnails/3.jpg)
Outline
• Background and motivation
• Finding precision bugs
• Finding soundness bugs
• Finding consistency bugs
• Application to Rust
• Results
![Page 4: Fuzzing the Rust Typechecker Using CLP - GitHub Pages · Fuzzing the Rust Typechecker Using CLP Kyle Dewey, Jared Roesch, Ben Hardekopf University of California, Santa Barbara. Teaser](https://reader030.vdocuments.site/reader030/viewer/2022040410/5ecccf5ec221095fc21e2b58/html5/thumbnails/4.jpg)
Outline
• Background and motivation
• Finding precision bugs
• Finding soundness bugs
• Finding consistency bugs
• Application to Rust
• Results
![Page 5: Fuzzing the Rust Typechecker Using CLP - GitHub Pages · Fuzzing the Rust Typechecker Using CLP Kyle Dewey, Jared Roesch, Ben Hardekopf University of California, Santa Barbara. Teaser](https://reader030.vdocuments.site/reader030/viewer/2022040410/5ecccf5ec221095fc21e2b58/html5/thumbnails/5.jpg)
Motivation• Typecheckers are crucial components in
statically typed languages
• Help ensure programs are correct
• Defend against exploits
P1
P2
Typechecker
P1
P2
3
![Page 6: Fuzzing the Rust Typechecker Using CLP - GitHub Pages · Fuzzing the Rust Typechecker Using CLP Kyle Dewey, Jared Roesch, Ben Hardekopf University of California, Santa Barbara. Teaser](https://reader030.vdocuments.site/reader030/viewer/2022040410/5ecccf5ec221095fc21e2b58/html5/thumbnails/6.jpg)
Motivation• Problem: typecheckers can be buggy too
• Fail to accept well-typed programs
• Fail to reject ill-typed programs
P1
P2
Typechecker
P2
4
![Page 7: Fuzzing the Rust Typechecker Using CLP - GitHub Pages · Fuzzing the Rust Typechecker Using CLP Kyle Dewey, Jared Roesch, Ben Hardekopf University of California, Santa Barbara. Teaser](https://reader030.vdocuments.site/reader030/viewer/2022040410/5ecccf5ec221095fc21e2b58/html5/thumbnails/7.jpg)
Idea
• Use black-box language fuzzing techniques to automatically find these bugs, ideally before they become a problem
P1
P2
Typechecker
P1
P2
5
![Page 8: Fuzzing the Rust Typechecker Using CLP - GitHub Pages · Fuzzing the Rust Typechecker Using CLP Kyle Dewey, Jared Roesch, Ben Hardekopf University of California, Santa Barbara. Teaser](https://reader030.vdocuments.site/reader030/viewer/2022040410/5ecccf5ec221095fc21e2b58/html5/thumbnails/8.jpg)
Existing Work
• Most existing work on language fuzzing fundamentally applies only to dynamically-typed languages (e.g., jsfunfuzz)
• Based on performing a random walk over the language’s grammar, referred to as a stochastic grammar
6
![Page 9: Fuzzing the Rust Typechecker Using CLP - GitHub Pages · Fuzzing the Rust Typechecker Using CLP Kyle Dewey, Jared Roesch, Ben Hardekopf University of California, Santa Barbara. Teaser](https://reader030.vdocuments.site/reader030/viewer/2022040410/5ecccf5ec221095fc21e2b58/html5/thumbnails/9.jpg)
Stochastic Grammars
7
![Page 10: Fuzzing the Rust Typechecker Using CLP - GitHub Pages · Fuzzing the Rust Typechecker Using CLP Kyle Dewey, Jared Roesch, Ben Hardekopf University of California, Santa Barbara. Teaser](https://reader030.vdocuments.site/reader030/viewer/2022040410/5ecccf5ec221095fc21e2b58/html5/thumbnails/10.jpg)
Stochastic Grammars
7
![Page 11: Fuzzing the Rust Typechecker Using CLP - GitHub Pages · Fuzzing the Rust Typechecker Using CLP Kyle Dewey, Jared Roesch, Ben Hardekopf University of California, Santa Barbara. Teaser](https://reader030.vdocuments.site/reader030/viewer/2022040410/5ecccf5ec221095fc21e2b58/html5/thumbnails/11.jpg)
Stochastic Grammars
7
![Page 12: Fuzzing the Rust Typechecker Using CLP - GitHub Pages · Fuzzing the Rust Typechecker Using CLP Kyle Dewey, Jared Roesch, Ben Hardekopf University of California, Santa Barbara. Teaser](https://reader030.vdocuments.site/reader030/viewer/2022040410/5ecccf5ec221095fc21e2b58/html5/thumbnails/12.jpg)
Stochastic Grammars0.3 0.7
0.4 0.4 0.2
7
![Page 13: Fuzzing the Rust Typechecker Using CLP - GitHub Pages · Fuzzing the Rust Typechecker Using CLP Kyle Dewey, Jared Roesch, Ben Hardekopf University of California, Santa Barbara. Teaser](https://reader030.vdocuments.site/reader030/viewer/2022040410/5ecccf5ec221095fc21e2b58/html5/thumbnails/13.jpg)
Stochastic Grammars0.3 0.7
0.4 0.4 0.2
e = 1
7
![Page 14: Fuzzing the Rust Typechecker Using CLP - GitHub Pages · Fuzzing the Rust Typechecker Using CLP Kyle Dewey, Jared Roesch, Ben Hardekopf University of California, Santa Barbara. Teaser](https://reader030.vdocuments.site/reader030/viewer/2022040410/5ecccf5ec221095fc21e2b58/html5/thumbnails/14.jpg)
Stochastic Grammars0.3 0.7
0.4 0.4 0.2
e = _ _ _
7
![Page 15: Fuzzing the Rust Typechecker Using CLP - GitHub Pages · Fuzzing the Rust Typechecker Using CLP Kyle Dewey, Jared Roesch, Ben Hardekopf University of California, Santa Barbara. Teaser](https://reader030.vdocuments.site/reader030/viewer/2022040410/5ecccf5ec221095fc21e2b58/html5/thumbnails/15.jpg)
Stochastic Grammars0.3 0.7
0.4 0.4 0.2
e = true _ _
7
![Page 16: Fuzzing the Rust Typechecker Using CLP - GitHub Pages · Fuzzing the Rust Typechecker Using CLP Kyle Dewey, Jared Roesch, Ben Hardekopf University of California, Santa Barbara. Teaser](https://reader030.vdocuments.site/reader030/viewer/2022040410/5ecccf5ec221095fc21e2b58/html5/thumbnails/16.jpg)
Stochastic Grammars0.3 0.7
0.4 0.4 0.2
e = true && _
7
![Page 17: Fuzzing the Rust Typechecker Using CLP - GitHub Pages · Fuzzing the Rust Typechecker Using CLP Kyle Dewey, Jared Roesch, Ben Hardekopf University of California, Santa Barbara. Teaser](https://reader030.vdocuments.site/reader030/viewer/2022040410/5ecccf5ec221095fc21e2b58/html5/thumbnails/17.jpg)
Stochastic Grammars0.3 0.7
0.4 0.4 0.2
e = true && true
7
![Page 18: Fuzzing the Rust Typechecker Using CLP - GitHub Pages · Fuzzing the Rust Typechecker Using CLP Kyle Dewey, Jared Roesch, Ben Hardekopf University of California, Santa Barbara. Teaser](https://reader030.vdocuments.site/reader030/viewer/2022040410/5ecccf5ec221095fc21e2b58/html5/thumbnails/18.jpg)
Stochastic Grammars0.3 0.7
0.4 0.4 0.2
e = _ _ _
7
![Page 19: Fuzzing the Rust Typechecker Using CLP - GitHub Pages · Fuzzing the Rust Typechecker Using CLP Kyle Dewey, Jared Roesch, Ben Hardekopf University of California, Santa Barbara. Teaser](https://reader030.vdocuments.site/reader030/viewer/2022040410/5ecccf5ec221095fc21e2b58/html5/thumbnails/19.jpg)
Stochastic Grammars0.3 0.7
0.4 0.4 0.2
e = 1 _ _
7
![Page 20: Fuzzing the Rust Typechecker Using CLP - GitHub Pages · Fuzzing the Rust Typechecker Using CLP Kyle Dewey, Jared Roesch, Ben Hardekopf University of California, Santa Barbara. Teaser](https://reader030.vdocuments.site/reader030/viewer/2022040410/5ecccf5ec221095fc21e2b58/html5/thumbnails/20.jpg)
Stochastic Grammars0.3 0.7
0.4 0.4 0.2
e = 1 + _
7
![Page 21: Fuzzing the Rust Typechecker Using CLP - GitHub Pages · Fuzzing the Rust Typechecker Using CLP Kyle Dewey, Jared Roesch, Ben Hardekopf University of California, Santa Barbara. Teaser](https://reader030.vdocuments.site/reader030/viewer/2022040410/5ecccf5ec221095fc21e2b58/html5/thumbnails/21.jpg)
Stochastic Grammars0.3 0.7
0.4 0.4 0.2
e = 1 + true
7
![Page 22: Fuzzing the Rust Typechecker Using CLP - GitHub Pages · Fuzzing the Rust Typechecker Using CLP Kyle Dewey, Jared Roesch, Ben Hardekopf University of California, Santa Barbara. Teaser](https://reader030.vdocuments.site/reader030/viewer/2022040410/5ecccf5ec221095fc21e2b58/html5/thumbnails/22.jpg)
Type Errors
• Dynamic setting - still a valid test!
• Static setting - tests if typechecker correctly rejects things. Except...
• No ground truth
• Most type errors are trivial
• Most randomly generated programs contain lots of type errors, which can mask each other
e = 1 + true
8
![Page 23: Fuzzing the Rust Typechecker Using CLP - GitHub Pages · Fuzzing the Rust Typechecker Using CLP Kyle Dewey, Jared Roesch, Ben Hardekopf University of California, Santa Barbara. Teaser](https://reader030.vdocuments.site/reader030/viewer/2022040410/5ecccf5ec221095fc21e2b58/html5/thumbnails/23.jpg)
Existing Solutions• All existing solutions that address these concerns
suffer from at least one of the following problems:
• Some generated programs are “accidentally” ill-typed
• Not all well-typed programs can be generated
• Fundamentally cannot handle the entire type system
• Highly specific to language being tested
9
![Page 24: Fuzzing the Rust Typechecker Using CLP - GitHub Pages · Fuzzing the Rust Typechecker Using CLP Kyle Dewey, Jared Roesch, Ben Hardekopf University of California, Santa Barbara. Teaser](https://reader030.vdocuments.site/reader030/viewer/2022040410/5ecccf5ec221095fc21e2b58/html5/thumbnails/24.jpg)
Existing Solutions
• In all cases, the typechecker is an adversary to be overcome in order to test downstream components
• All implicitly assume the typechecker is correct
10
![Page 25: Fuzzing the Rust Typechecker Using CLP - GitHub Pages · Fuzzing the Rust Typechecker Using CLP Kyle Dewey, Jared Roesch, Ben Hardekopf University of California, Santa Barbara. Teaser](https://reader030.vdocuments.site/reader030/viewer/2022040410/5ecccf5ec221095fc21e2b58/html5/thumbnails/25.jpg)
Our Approach
![Page 26: Fuzzing the Rust Typechecker Using CLP - GitHub Pages · Fuzzing the Rust Typechecker Using CLP Kyle Dewey, Jared Roesch, Ben Hardekopf University of California, Santa Barbara. Teaser](https://reader030.vdocuments.site/reader030/viewer/2022040410/5ecccf5ec221095fc21e2b58/html5/thumbnails/26.jpg)
Our Approach• We focus our testing efforts on finding
three specific kinds of typechecker bugs:
• Failure to accept a well-typed program
• Failure to reject an ill-typed program
• Inconsistent behavior on type equivalent programs
• We have devised general techniques for finding these three kinds of bugs
11
![Page 27: Fuzzing the Rust Typechecker Using CLP - GitHub Pages · Fuzzing the Rust Typechecker Using CLP Kyle Dewey, Jared Roesch, Ben Hardekopf University of California, Santa Barbara. Teaser](https://reader030.vdocuments.site/reader030/viewer/2022040410/5ecccf5ec221095fc21e2b58/html5/thumbnails/27.jpg)
Viewpoint from Program Analysis (1)
• Failure to accept a well-typed program is a precision bug
• While annoying to the programmer, guarantees provided by the type system are preserved
12
![Page 28: Fuzzing the Rust Typechecker Using CLP - GitHub Pages · Fuzzing the Rust Typechecker Using CLP Kyle Dewey, Jared Roesch, Ben Hardekopf University of California, Santa Barbara. Teaser](https://reader030.vdocuments.site/reader030/viewer/2022040410/5ecccf5ec221095fc21e2b58/html5/thumbnails/28.jpg)
• Failure to reject an ill-typed program is a soundness bug
• Silent loss of guarantees provided by the type system
• Potentially devastating
Viewpoint from Program Analysis (2)
13
![Page 29: Fuzzing the Rust Typechecker Using CLP - GitHub Pages · Fuzzing the Rust Typechecker Using CLP Kyle Dewey, Jared Roesch, Ben Hardekopf University of California, Santa Barbara. Teaser](https://reader030.vdocuments.site/reader030/viewer/2022040410/5ecccf5ec221095fc21e2b58/html5/thumbnails/29.jpg)
Outline
• Background and motivation
• Finding precision bugs
• Finding soundness bugs
• Finding consistency bugs
• Application to Rust
• Results
![Page 30: Fuzzing the Rust Typechecker Using CLP - GitHub Pages · Fuzzing the Rust Typechecker Using CLP Kyle Dewey, Jared Roesch, Ben Hardekopf University of California, Santa Barbara. Teaser](https://reader030.vdocuments.site/reader030/viewer/2022040410/5ecccf5ec221095fc21e2b58/html5/thumbnails/30.jpg)
Finding Precision Bugs
• Intuition: generate guaranteed well-typed programs
• Any rejected programs indicate bugs
14
![Page 31: Fuzzing the Rust Typechecker Using CLP - GitHub Pages · Fuzzing the Rust Typechecker Using CLP Kyle Dewey, Jared Roesch, Ben Hardekopf University of California, Santa Barbara. Teaser](https://reader030.vdocuments.site/reader030/viewer/2022040410/5ecccf5ec221095fc21e2b58/html5/thumbnails/31.jpg)
Generating Well-Typed Programs
• We use constraint logic programming (CLP) for this purpose
• Typing rules can be specified in CLP, and CLP engines can execute them “backwards” to generate programs which are well-typed
15
![Page 32: Fuzzing the Rust Typechecker Using CLP - GitHub Pages · Fuzzing the Rust Typechecker Using CLP Kyle Dewey, Jared Roesch, Ben Hardekopf University of California, Santa Barbara. Teaser](https://reader030.vdocuments.site/reader030/viewer/2022040410/5ecccf5ec221095fc21e2b58/html5/thumbnails/32.jpg)
Well-Typed Generation Example: System F
![Page 33: Fuzzing the Rust Typechecker Using CLP - GitHub Pages · Fuzzing the Rust Typechecker Using CLP Kyle Dewey, Jared Roesch, Ben Hardekopf University of California, Santa Barbara. Teaser](https://reader030.vdocuments.site/reader030/viewer/2022040410/5ecccf5ec221095fc21e2b58/html5/thumbnails/33.jpg)
System F Highlights
• This is the simply-typed lambda calculus...
• Higher-order functions
• ...with parametric polymorphism
• Type variables
• Serves as a simple example
• Despite simplicity, both higher-order functions and type variables fundamentally cannot be handled by prior work
16
![Page 34: Fuzzing the Rust Typechecker Using CLP - GitHub Pages · Fuzzing the Rust Typechecker Using CLP Kyle Dewey, Jared Roesch, Ben Hardekopf University of California, Santa Barbara. Teaser](https://reader030.vdocuments.site/reader030/viewer/2022040410/5ecccf5ec221095fc21e2b58/html5/thumbnails/34.jpg)
Grammar and Types
17
![Page 35: Fuzzing the Rust Typechecker Using CLP - GitHub Pages · Fuzzing the Rust Typechecker Using CLP Kyle Dewey, Jared Roesch, Ben Hardekopf University of California, Santa Barbara. Teaser](https://reader030.vdocuments.site/reader030/viewer/2022040410/5ecccf5ec221095fc21e2b58/html5/thumbnails/35.jpg)
Grammar and Types
17
![Page 36: Fuzzing the Rust Typechecker Using CLP - GitHub Pages · Fuzzing the Rust Typechecker Using CLP Kyle Dewey, Jared Roesch, Ben Hardekopf University of California, Santa Barbara. Teaser](https://reader030.vdocuments.site/reader030/viewer/2022040410/5ecccf5ec221095fc21e2b58/html5/thumbnails/36.jpg)
Grammar and Types
17
![Page 37: Fuzzing the Rust Typechecker Using CLP - GitHub Pages · Fuzzing the Rust Typechecker Using CLP Kyle Dewey, Jared Roesch, Ben Hardekopf University of California, Santa Barbara. Teaser](https://reader030.vdocuments.site/reader030/viewer/2022040410/5ecccf5ec221095fc21e2b58/html5/thumbnails/37.jpg)
Grammar and Types
17
![Page 38: Fuzzing the Rust Typechecker Using CLP - GitHub Pages · Fuzzing the Rust Typechecker Using CLP Kyle Dewey, Jared Roesch, Ben Hardekopf University of California, Santa Barbara. Teaser](https://reader030.vdocuments.site/reader030/viewer/2022040410/5ecccf5ec221095fc21e2b58/html5/thumbnails/38.jpg)
Grammar and Types
17
![Page 39: Fuzzing the Rust Typechecker Using CLP - GitHub Pages · Fuzzing the Rust Typechecker Using CLP Kyle Dewey, Jared Roesch, Ben Hardekopf University of California, Santa Barbara. Teaser](https://reader030.vdocuments.site/reader030/viewer/2022040410/5ecccf5ec221095fc21e2b58/html5/thumbnails/39.jpg)
Grammar and Types
17
![Page 40: Fuzzing the Rust Typechecker Using CLP - GitHub Pages · Fuzzing the Rust Typechecker Using CLP Kyle Dewey, Jared Roesch, Ben Hardekopf University of California, Santa Barbara. Teaser](https://reader030.vdocuments.site/reader030/viewer/2022040410/5ecccf5ec221095fc21e2b58/html5/thumbnails/40.jpg)
Grammar and Types
17
![Page 41: Fuzzing the Rust Typechecker Using CLP - GitHub Pages · Fuzzing the Rust Typechecker Using CLP Kyle Dewey, Jared Roesch, Ben Hardekopf University of California, Santa Barbara. Teaser](https://reader030.vdocuments.site/reader030/viewer/2022040410/5ecccf5ec221095fc21e2b58/html5/thumbnails/41.jpg)
Grammar and Types
17
![Page 42: Fuzzing the Rust Typechecker Using CLP - GitHub Pages · Fuzzing the Rust Typechecker Using CLP Kyle Dewey, Jared Roesch, Ben Hardekopf University of California, Santa Barbara. Teaser](https://reader030.vdocuments.site/reader030/viewer/2022040410/5ecccf5ec221095fc21e2b58/html5/thumbnails/42.jpg)
Grammar and Types
17
![Page 43: Fuzzing the Rust Typechecker Using CLP - GitHub Pages · Fuzzing the Rust Typechecker Using CLP Kyle Dewey, Jared Roesch, Ben Hardekopf University of California, Santa Barbara. Teaser](https://reader030.vdocuments.site/reader030/viewer/2022040410/5ecccf5ec221095fc21e2b58/html5/thumbnails/43.jpg)
Typing Rules in CLP
typing(Gamma, var(X), T) :- lookup(Gamma, X, T).
Typing RuleCLP Code
18
![Page 44: Fuzzing the Rust Typechecker Using CLP - GitHub Pages · Fuzzing the Rust Typechecker Using CLP Kyle Dewey, Jared Roesch, Ben Hardekopf University of California, Santa Barbara. Teaser](https://reader030.vdocuments.site/reader030/viewer/2022040410/5ecccf5ec221095fc21e2b58/html5/thumbnails/44.jpg)
Typing Rules in CLP
typing(Gamma, var(X), T) :- lookup(Gamma, X, T).
Typing RuleCLP Code
18
![Page 45: Fuzzing the Rust Typechecker Using CLP - GitHub Pages · Fuzzing the Rust Typechecker Using CLP Kyle Dewey, Jared Roesch, Ben Hardekopf University of California, Santa Barbara. Teaser](https://reader030.vdocuments.site/reader030/viewer/2022040410/5ecccf5ec221095fc21e2b58/html5/thumbnails/45.jpg)
Typing Rules in CLP
typing(Gamma, var(X), T) :- lookup(Gamma, X, T).
Typing RuleCLP Code
18
![Page 46: Fuzzing the Rust Typechecker Using CLP - GitHub Pages · Fuzzing the Rust Typechecker Using CLP Kyle Dewey, Jared Roesch, Ben Hardekopf University of California, Santa Barbara. Teaser](https://reader030.vdocuments.site/reader030/viewer/2022040410/5ecccf5ec221095fc21e2b58/html5/thumbnails/46.jpg)
Typing Rules in CLP
typing(Gamma, var(X), T) :- lookup(Gamma, X, T).
Typing RuleCLP Code
18
![Page 47: Fuzzing the Rust Typechecker Using CLP - GitHub Pages · Fuzzing the Rust Typechecker Using CLP Kyle Dewey, Jared Roesch, Ben Hardekopf University of California, Santa Barbara. Teaser](https://reader030.vdocuments.site/reader030/viewer/2022040410/5ecccf5ec221095fc21e2b58/html5/thumbnails/47.jpg)
Typing Rules in CLP
typing(Gamma, var(X), T) :- lookup(Gamma, X, T).
Typing RuleCLP Code
18
![Page 48: Fuzzing the Rust Typechecker Using CLP - GitHub Pages · Fuzzing the Rust Typechecker Using CLP Kyle Dewey, Jared Roesch, Ben Hardekopf University of California, Santa Barbara. Teaser](https://reader030.vdocuments.site/reader030/viewer/2022040410/5ecccf5ec221095fc21e2b58/html5/thumbnails/48.jpg)
Typing Rules in CLP
typing(Gamma, app(E1, E2), T2) :- typing(Gamma, E1, arrow(T1, T2)), typing(Gamma, E2, T1).
Typing RuleCLP Code
19
![Page 49: Fuzzing the Rust Typechecker Using CLP - GitHub Pages · Fuzzing the Rust Typechecker Using CLP Kyle Dewey, Jared Roesch, Ben Hardekopf University of California, Santa Barbara. Teaser](https://reader030.vdocuments.site/reader030/viewer/2022040410/5ecccf5ec221095fc21e2b58/html5/thumbnails/49.jpg)
Typing Rules in CLP
typing(Gamma, app(E1, E2), T2) :- typing(Gamma, E1, arrow(T1, T2)), typing(Gamma, E2, T1).
Typing RuleCLP Code
19
![Page 50: Fuzzing the Rust Typechecker Using CLP - GitHub Pages · Fuzzing the Rust Typechecker Using CLP Kyle Dewey, Jared Roesch, Ben Hardekopf University of California, Santa Barbara. Teaser](https://reader030.vdocuments.site/reader030/viewer/2022040410/5ecccf5ec221095fc21e2b58/html5/thumbnails/50.jpg)
Typing Rules in CLP
typing(Gamma, app(E1, E2), T2) :- typing(Gamma, E1, arrow(T1, T2)), typing(Gamma, E2, T1).
Typing RuleCLP Code
19
![Page 51: Fuzzing the Rust Typechecker Using CLP - GitHub Pages · Fuzzing the Rust Typechecker Using CLP Kyle Dewey, Jared Roesch, Ben Hardekopf University of California, Santa Barbara. Teaser](https://reader030.vdocuments.site/reader030/viewer/2022040410/5ecccf5ec221095fc21e2b58/html5/thumbnails/51.jpg)
Typing Rules in CLP
typing(Gamma, app(E1, E2), T2) :- typing(Gamma, E1, arrow(T1, T2)), typing(Gamma, E2, T1).
Typing RuleCLP Code
19
![Page 52: Fuzzing the Rust Typechecker Using CLP - GitHub Pages · Fuzzing the Rust Typechecker Using CLP Kyle Dewey, Jared Roesch, Ben Hardekopf University of California, Santa Barbara. Teaser](https://reader030.vdocuments.site/reader030/viewer/2022040410/5ecccf5ec221095fc21e2b58/html5/thumbnails/52.jpg)
Typing Rules in CLP
typing(Gamma, app(E1, E2), T2) :- typing(Gamma, E1, arrow(T1, T2)), typing(Gamma, E2, T1).
Typing RuleCLP Code
19
![Page 53: Fuzzing the Rust Typechecker Using CLP - GitHub Pages · Fuzzing the Rust Typechecker Using CLP Kyle Dewey, Jared Roesch, Ben Hardekopf University of California, Santa Barbara. Teaser](https://reader030.vdocuments.site/reader030/viewer/2022040410/5ecccf5ec221095fc21e2b58/html5/thumbnails/53.jpg)
Typing Rules in CLP
typing(Gamma, app(E1, E2), T2) :- typing(Gamma, E1, arrow(T1, T2)), typing(Gamma, E2, T1).
Typing RuleCLP Code
19
![Page 54: Fuzzing the Rust Typechecker Using CLP - GitHub Pages · Fuzzing the Rust Typechecker Using CLP Kyle Dewey, Jared Roesch, Ben Hardekopf University of California, Santa Barbara. Teaser](https://reader030.vdocuments.site/reader030/viewer/2022040410/5ecccf5ec221095fc21e2b58/html5/thumbnails/54.jpg)
From Typing Rules to a Generator
• We have implemented a typechecker here
• This can be trivially turned into a generator of well-typed terms, like so (where ?- indicates what to execute):
?- typing([], E, T), write(E), fail.
20
![Page 55: Fuzzing the Rust Typechecker Using CLP - GitHub Pages · Fuzzing the Rust Typechecker Using CLP Kyle Dewey, Jared Roesch, Ben Hardekopf University of California, Santa Barbara. Teaser](https://reader030.vdocuments.site/reader030/viewer/2022040410/5ecccf5ec221095fc21e2b58/html5/thumbnails/55.jpg)
From Typing Rules to a Generator
• We have implemented a typechecker here
• This can be trivially turned into a generator of well-typed terms, like so (where ?- indicates what to execute):
?- typing([], E, T), write(E), fail.
20
![Page 56: Fuzzing the Rust Typechecker Using CLP - GitHub Pages · Fuzzing the Rust Typechecker Using CLP Kyle Dewey, Jared Roesch, Ben Hardekopf University of California, Santa Barbara. Teaser](https://reader030.vdocuments.site/reader030/viewer/2022040410/5ecccf5ec221095fc21e2b58/html5/thumbnails/56.jpg)
From Typing Rules to a Generator
• We have implemented a typechecker here
• This can be trivially turned into a generator of well-typed terms, like so (where ?- indicates what to execute):
?- typing([], E, T), write(E), fail.
20
![Page 57: Fuzzing the Rust Typechecker Using CLP - GitHub Pages · Fuzzing the Rust Typechecker Using CLP Kyle Dewey, Jared Roesch, Ben Hardekopf University of California, Santa Barbara. Teaser](https://reader030.vdocuments.site/reader030/viewer/2022040410/5ecccf5ec221095fc21e2b58/html5/thumbnails/57.jpg)
From Typing Rules to a Generator
• We have implemented a typechecker here
• This can be trivially turned into a generator of well-typed terms, like so (where ?- indicates what to execute):
?- typing([], E, T), write(E), fail.
20
![Page 58: Fuzzing the Rust Typechecker Using CLP - GitHub Pages · Fuzzing the Rust Typechecker Using CLP Kyle Dewey, Jared Roesch, Ben Hardekopf University of California, Santa Barbara. Teaser](https://reader030.vdocuments.site/reader030/viewer/2022040410/5ecccf5ec221095fc21e2b58/html5/thumbnails/58.jpg)
Take-Home Point
• This generator of well-typed terms can be used to find precision bugs in typecheckers
• Since everything generated is well-typed, if anything is rejected, it indicates the typechecker under test is buggy under the particular input
21
![Page 59: Fuzzing the Rust Typechecker Using CLP - GitHub Pages · Fuzzing the Rust Typechecker Using CLP Kyle Dewey, Jared Roesch, Ben Hardekopf University of California, Santa Barbara. Teaser](https://reader030.vdocuments.site/reader030/viewer/2022040410/5ecccf5ec221095fc21e2b58/html5/thumbnails/59.jpg)
Outline
• Background and motivation
• Finding precision bugs
• Finding soundness bugs
• Finding consistency bugs
• Application to Rust
• Results
![Page 60: Fuzzing the Rust Typechecker Using CLP - GitHub Pages · Fuzzing the Rust Typechecker Using CLP Kyle Dewey, Jared Roesch, Ben Hardekopf University of California, Santa Barbara. Teaser](https://reader030.vdocuments.site/reader030/viewer/2022040410/5ecccf5ec221095fc21e2b58/html5/thumbnails/60.jpg)
Finding Soundness Bugs
• Intuition: generate ill-typed programs
• If the typechecker accepts any of them, then we have discovered a bug
• Simple solution: generate syntactically valid programs, and filter out those that happen to be well-typed (which occur rarely)
22
![Page 61: Fuzzing the Rust Typechecker Using CLP - GitHub Pages · Fuzzing the Rust Typechecker Using CLP Kyle Dewey, Jared Roesch, Ben Hardekopf University of California, Santa Barbara. Teaser](https://reader030.vdocuments.site/reader030/viewer/2022040410/5ecccf5ec221095fc21e2b58/html5/thumbnails/61.jpg)
Finding Soundness Bugs
• A purely syntactic approach results in fairly uninteresting tests
• They do not exploit information about the underlying type system
• Tend to be obviously ill-typed, so intuitively only the buggiest of typecheckers would let them through
23
![Page 62: Fuzzing the Rust Typechecker Using CLP - GitHub Pages · Fuzzing the Rust Typechecker Using CLP Kyle Dewey, Jared Roesch, Ben Hardekopf University of California, Santa Barbara. Teaser](https://reader030.vdocuments.site/reader030/viewer/2022040410/5ecccf5ec221095fc21e2b58/html5/thumbnails/62.jpg)
Better Approach
• Generate almost well-typed programs, which are ill-typed, but in subtle ways
• Intuitively, one simply negates a single premise of a single typing rule, in a nondeterministic manner
• Based on developer feedback
24
![Page 63: Fuzzing the Rust Typechecker Using CLP - GitHub Pages · Fuzzing the Rust Typechecker Using CLP Kyle Dewey, Jared Roesch, Ben Hardekopf University of California, Santa Barbara. Teaser](https://reader030.vdocuments.site/reader030/viewer/2022040410/5ecccf5ec221095fc21e2b58/html5/thumbnails/63.jpg)
Almost Well-Typed Generation Example:
System F
![Page 64: Fuzzing the Rust Typechecker Using CLP - GitHub Pages · Fuzzing the Rust Typechecker Using CLP Kyle Dewey, Jared Roesch, Ben Hardekopf University of California, Santa Barbara. Teaser](https://reader030.vdocuments.site/reader030/viewer/2022040410/5ecccf5ec221095fc21e2b58/html5/thumbnails/64.jpg)
Typing Rules in CLP
typing(Gamma, app(E1, E2), T2) :- typing(Gamma, E1, arrow(T1, T2)), typing(Gamma, E2, T1).
Typing RuleCLP Code
25
![Page 65: Fuzzing the Rust Typechecker Using CLP - GitHub Pages · Fuzzing the Rust Typechecker Using CLP Kyle Dewey, Jared Roesch, Ben Hardekopf University of California, Santa Barbara. Teaser](https://reader030.vdocuments.site/reader030/viewer/2022040410/5ecccf5ec221095fc21e2b58/html5/thumbnails/65.jpg)
Typing Rules in CLP
Typing RuleCLP Code
typing(Gamma, app(E1, E2), T2) :- typing(Gamma, E1, arrow(T1, T2)), typing(Gamma, E2, T3), T3 \== T1.
25
![Page 66: Fuzzing the Rust Typechecker Using CLP - GitHub Pages · Fuzzing the Rust Typechecker Using CLP Kyle Dewey, Jared Roesch, Ben Hardekopf University of California, Santa Barbara. Teaser](https://reader030.vdocuments.site/reader030/viewer/2022040410/5ecccf5ec221095fc21e2b58/html5/thumbnails/66.jpg)
Outline
• Background and motivation
• Finding precision bugs
• Finding soundness bugs
• Finding consistency bugs
• Application to Rust
• Results
![Page 67: Fuzzing the Rust Typechecker Using CLP - GitHub Pages · Fuzzing the Rust Typechecker Using CLP Kyle Dewey, Jared Roesch, Ben Hardekopf University of California, Santa Barbara. Teaser](https://reader030.vdocuments.site/reader030/viewer/2022040410/5ecccf5ec221095fc21e2b58/html5/thumbnails/67.jpg)
Why Another Type of Bug?
• Theoretically, soundness and precision covers the entire state space
• Finding all possible precision and soundness bugs requires a full-blown typechecker implemented in CLP
• Lots of work
• Depending on the language, ground truth may be unclear
26
![Page 68: Fuzzing the Rust Typechecker Using CLP - GitHub Pages · Fuzzing the Rust Typechecker Using CLP Kyle Dewey, Jared Roesch, Ben Hardekopf University of California, Santa Barbara. Teaser](https://reader030.vdocuments.site/reader030/viewer/2022040410/5ecccf5ec221095fc21e2b58/html5/thumbnails/68.jpg)
Consistency Bugs
• Advantage: full ground truth is not necessary, only an understanding of what constitutes a type equivalent program
• This is generally much simpler
• If the typechecker behaves differently on type equivalent programs, it indicates a bug
• Both should be either well-typed or ill-typed
27
![Page 69: Fuzzing the Rust Typechecker Using CLP - GitHub Pages · Fuzzing the Rust Typechecker Using CLP Kyle Dewey, Jared Roesch, Ben Hardekopf University of California, Santa Barbara. Teaser](https://reader030.vdocuments.site/reader030/viewer/2022040410/5ecccf5ec221095fc21e2b58/html5/thumbnails/69.jpg)
Implementing Consistency Bug Finders
• Basic idea: write a syntax-based generator, using traditional fuzzing techniques
• Pass the output of this generator through a series of rewrite rules
• Ensure that both the input and the output to the rewrite rules behave the same
28
![Page 70: Fuzzing the Rust Typechecker Using CLP - GitHub Pages · Fuzzing the Rust Typechecker Using CLP Kyle Dewey, Jared Roesch, Ben Hardekopf University of California, Santa Barbara. Teaser](https://reader030.vdocuments.site/reader030/viewer/2022040410/5ecccf5ec221095fc21e2b58/html5/thumbnails/70.jpg)
Outline
• Background and motivation
• Finding precision bugs
• Finding soundness bugs
• Finding consistency bugs
• Application to Rust
• Results
![Page 71: Fuzzing the Rust Typechecker Using CLP - GitHub Pages · Fuzzing the Rust Typechecker Using CLP Kyle Dewey, Jared Roesch, Ben Hardekopf University of California, Santa Barbara. Teaser](https://reader030.vdocuments.site/reader030/viewer/2022040410/5ecccf5ec221095fc21e2b58/html5/thumbnails/71.jpg)
Why Rust?• A real language with a rapidly growing user
base (over 3,300 packages available)
• A sophisticated type system with important guarantees (e.g., memory safety without GC)
• No formal semantics, or even an informal specification
• Worked closely with Rust development team
29
![Page 72: Fuzzing the Rust Typechecker Using CLP - GitHub Pages · Fuzzing the Rust Typechecker Using CLP Kyle Dewey, Jared Roesch, Ben Hardekopf University of California, Santa Barbara. Teaser](https://reader030.vdocuments.site/reader030/viewer/2022040410/5ecccf5ec221095fc21e2b58/html5/thumbnails/72.jpg)
Key Rust Type System Features
• Parametric polymorphism
• Generics
• Typeclasses
• Associated types
• Affine types
• Borrowing (reference types)
30
![Page 73: Fuzzing the Rust Typechecker Using CLP - GitHub Pages · Fuzzing the Rust Typechecker Using CLP Kyle Dewey, Jared Roesch, Ben Hardekopf University of California, Santa Barbara. Teaser](https://reader030.vdocuments.site/reader030/viewer/2022040410/5ecccf5ec221095fc21e2b58/html5/thumbnails/73.jpg)
Testing Methodology
• Handling all of the language with one fuzzer is extremely difficult
• Simpler approach: develop a series of fuzzers which handle subsets of the language
• Use different techniques for each fuzzer
31
![Page 74: Fuzzing the Rust Typechecker Using CLP - GitHub Pages · Fuzzing the Rust Typechecker Using CLP Kyle Dewey, Jared Roesch, Ben Hardekopf University of California, Santa Barbara. Teaser](https://reader030.vdocuments.site/reader030/viewer/2022040410/5ecccf5ec221095fc21e2b58/html5/thumbnails/74.jpg)
Outline
• Background and motivation
• Finding precision bugs
• Finding soundness bugs
• Finding consistency bugs
• Application to Rust
• Results
![Page 75: Fuzzing the Rust Typechecker Using CLP - GitHub Pages · Fuzzing the Rust Typechecker Using CLP Kyle Dewey, Jared Roesch, Ben Hardekopf University of California, Santa Barbara. Teaser](https://reader030.vdocuments.site/reader030/viewer/2022040410/5ecccf5ec221095fc21e2b58/html5/thumbnails/75.jpg)
Results
• 18 bugs found across all categories
• 14 confirmed by developers
• Includes a specification-level bug, where a program was legally considered both ill-typed and well-typed
• This work preceded a massive overhaul of the typechecker and overall type system
32
![Page 76: Fuzzing the Rust Typechecker Using CLP - GitHub Pages · Fuzzing the Rust Typechecker Using CLP Kyle Dewey, Jared Roesch, Ben Hardekopf University of California, Santa Barbara. Teaser](https://reader030.vdocuments.site/reader030/viewer/2022040410/5ecccf5ec221095fc21e2b58/html5/thumbnails/76.jpg)
Conclusions
• We identify three general kinds of typechecker bugs
• We describe automated techniques for finding each of these three kinds of bugs
• We apply these ideas to the Rust programming language, finding 14 confirmed bugs, all of which either have or are being addressed
33