is code review the solution?
TRANSCRIPT
![Page 1: Is code review the solution?](https://reader033.vdocuments.site/reader033/viewer/2022052509/55a0d35b1a28ab59108b47ef/html5/thumbnails/1.jpg)
Is Code Review the Solution?
Versão 1.1 - 28/10/2014
Confraria da Segurança da Informação
![Page 2: Is code review the solution?](https://reader033.vdocuments.site/reader033/viewer/2022052509/55a0d35b1a28ab59108b47ef/html5/thumbnails/2.jpg)
SAPO Websecurity Team
Outline
2
• What is code review • Mo9va9on • Open-‐Source • How to • Tools • Problems
![Page 3: Is code review the solution?](https://reader033.vdocuments.site/reader033/viewer/2022052509/55a0d35b1a28ab59108b47ef/html5/thumbnails/3.jpg)
SAPO Websecurity TeamSAPO Websecurity Team
About me
• Security Engineer at Portugal Telecom since 2004
– honeypots, traffic analysis, internal security
• At SAPO since 2010
– pentes9ng of web applica9ons, iOS, Android, IPTV
– all terrain security consultant
• Trainer of Linux and network security courses at Citeforma
• Speaker at security events like Codebits, Just4Mee9ng, Security Mee9ng, ISEL Tech, Create Tech, Confraria da Segurança da Informação and BSides Lisbon
• Holds a MSc in Informa9on Technology/Informa9on Security from Carnegie Mellon and CISSP
3
![Page 4: Is code review the solution?](https://reader033.vdocuments.site/reader033/viewer/2022052509/55a0d35b1a28ab59108b47ef/html5/thumbnails/4.jpg)
SAPO Websecurity Team
What is code review
• Code • Firefox -‐ 5 millions LOC (Lines of Code) • MySQL -‐ 12 millions LOC • Debian 5 -‐ 66 millions LOC • Windows Server 2003 -‐ 50 millions LOC
4
![Page 5: Is code review the solution?](https://reader033.vdocuments.site/reader033/viewer/2022052509/55a0d35b1a28ab59108b47ef/html5/thumbnails/5.jpg)
SAPO Websecurity Team
What is code review
• Review
• “formal assessment of something with the inten9on of ins9tu9ng change if necessary”
5
![Page 6: Is code review the solution?](https://reader033.vdocuments.site/reader033/viewer/2022052509/55a0d35b1a28ab59108b47ef/html5/thumbnails/6.jpg)
SAPO Websecurity Team
What is code review
• Code review is the analysis of source code in order to find defects – security, performance, func9onal, etc. –early detec9on – complements scanners and other tools
6
![Page 7: Is code review the solution?](https://reader033.vdocuments.site/reader033/viewer/2022052509/55a0d35b1a28ab59108b47ef/html5/thumbnails/7.jpg)
SAPO Websecurity Team
Motivation
7
![Page 8: Is code review the solution?](https://reader033.vdocuments.site/reader033/viewer/2022052509/55a0d35b1a28ab59108b47ef/html5/thumbnails/8.jpg)
SAPO Websecurity Team
Motivation
7
![Page 9: Is code review the solution?](https://reader033.vdocuments.site/reader033/viewer/2022052509/55a0d35b1a28ab59108b47ef/html5/thumbnails/9.jpg)
SAPO Websecurity Team
Motivation
7
![Page 10: Is code review the solution?](https://reader033.vdocuments.site/reader033/viewer/2022052509/55a0d35b1a28ab59108b47ef/html5/thumbnails/10.jpg)
SAPO Websecurity Team
Motivation
7
![Page 11: Is code review the solution?](https://reader033.vdocuments.site/reader033/viewer/2022052509/55a0d35b1a28ab59108b47ef/html5/thumbnails/11.jpg)
SAPO Websecurity Team
Motivation
7
![Page 12: Is code review the solution?](https://reader033.vdocuments.site/reader033/viewer/2022052509/55a0d35b1a28ab59108b47ef/html5/thumbnails/12.jpg)
SAPO Websecurity Team
Motivation
7
![Page 13: Is code review the solution?](https://reader033.vdocuments.site/reader033/viewer/2022052509/55a0d35b1a28ab59108b47ef/html5/thumbnails/13.jpg)
SAPO Websecurity Team
Motivation
8
![Page 14: Is code review the solution?](https://reader033.vdocuments.site/reader033/viewer/2022052509/55a0d35b1a28ab59108b47ef/html5/thumbnails/14.jpg)
SAPO Websecurity Team
Motivation
8
![Page 15: Is code review the solution?](https://reader033.vdocuments.site/reader033/viewer/2022052509/55a0d35b1a28ab59108b47ef/html5/thumbnails/15.jpg)
SAPO Websecurity Team
Motivation
8
![Page 16: Is code review the solution?](https://reader033.vdocuments.site/reader033/viewer/2022052509/55a0d35b1a28ab59108b47ef/html5/thumbnails/16.jpg)
SAPO Websecurity Team
Motivation
8
![Page 17: Is code review the solution?](https://reader033.vdocuments.site/reader033/viewer/2022052509/55a0d35b1a28ab59108b47ef/html5/thumbnails/17.jpg)
SAPO Websecurity Team
Motivation
8
![Page 18: Is code review the solution?](https://reader033.vdocuments.site/reader033/viewer/2022052509/55a0d35b1a28ab59108b47ef/html5/thumbnails/18.jpg)
SAPO Websecurity Team
Motivation
• Compliance
• PCI-‐DSS -‐ Payment Card Industry Data Security Standard • since 2005 • version 3.0
• Requirement 6.3.2 • “Review custom code prior to release to produc9on or customers in order to iden9fy any poten9al coding vulnerability (using either manual or automated processes) … “
9
![Page 19: Is code review the solution?](https://reader033.vdocuments.site/reader033/viewer/2022052509/55a0d35b1a28ab59108b47ef/html5/thumbnails/19.jpg)
SAPO Websecurity Team
Open-Source
• A requirement for code review is to have access to the source code
• Open-‐Source Sobware (OSS) makes its source code available for anyone (to review)
• Therefore, OSS is becer because its reviewed by the whole world • is it?
10
![Page 20: Is code review the solution?](https://reader033.vdocuments.site/reader033/viewer/2022052509/55a0d35b1a28ab59108b47ef/html5/thumbnails/20.jpg)
SAPO Websecurity Team
Open-Source
11
![Page 21: Is code review the solution?](https://reader033.vdocuments.site/reader033/viewer/2022052509/55a0d35b1a28ab59108b47ef/html5/thumbnails/21.jpg)
SAPO Websecurity Team
Open-Source
• Not all OSS is thoroughly reviewed, but…
12
![Page 22: Is code review the solution?](https://reader033.vdocuments.site/reader033/viewer/2022052509/55a0d35b1a28ab59108b47ef/html5/thumbnails/22.jpg)
SAPO Websecurity Team
Open-Source
• Not all OSS is thoroughly reviewed, but…
12
• In 2011, a vulnerability that allowed backup decryp9on was found
![Page 23: Is code review the solution?](https://reader033.vdocuments.site/reader033/viewer/2022052509/55a0d35b1a28ab59108b47ef/html5/thumbnails/23.jpg)
SAPO Websecurity Team
Open-Source
• Not all OSS is thoroughly reviewed, but…
12
• In 2011, a vulnerability that allowed backup decryp9on was found
![Page 24: Is code review the solution?](https://reader033.vdocuments.site/reader033/viewer/2022052509/55a0d35b1a28ab59108b47ef/html5/thumbnails/24.jpg)
SAPO Websecurity Team
Open-Source
• Not all OSS is thoroughly reviewed, but…
12
• In 2011, a vulnerability that allowed backup decryp9on was found
![Page 25: Is code review the solution?](https://reader033.vdocuments.site/reader033/viewer/2022052509/55a0d35b1a28ab59108b47ef/html5/thumbnails/25.jpg)
SAPO Websecurity Team
Open-Source
• Found “by someone who was reading the Tarsnap source code purely of curiosity”
• Lead to a bug bounty for security problems • “I'm a great fan of curiosity, but I've also learned that money can help to encourage curiosity.“
13
![Page 26: Is code review the solution?](https://reader033.vdocuments.site/reader033/viewer/2022052509/55a0d35b1a28ab59108b47ef/html5/thumbnails/26.jpg)
SAPO Websecurity Team
Open-Source
14
![Page 27: Is code review the solution?](https://reader033.vdocuments.site/reader033/viewer/2022052509/55a0d35b1a28ab59108b47ef/html5/thumbnails/27.jpg)
SAPO Websecurity Team
Open-Source
• Apple “goto fail”
• CVE-‐2014-‐1266 -‐ “acacker with a privileged network posi9on may capture or modify data in sessions protected by SSL/TLS”
• Affected iOS and OS X
• hcp://pi5.20.sl.pt
15
![Page 28: Is code review the solution?](https://reader033.vdocuments.site/reader033/viewer/2022052509/55a0d35b1a28ab59108b47ef/html5/thumbnails/28.jpg)
SAPO Websecurity Team
Open-Source
• Apple “goto fail”
16
![Page 29: Is code review the solution?](https://reader033.vdocuments.site/reader033/viewer/2022052509/55a0d35b1a28ab59108b47ef/html5/thumbnails/29.jpg)
SAPO Websecurity Team
Open-Source
• Apple “goto fail”
16
![Page 30: Is code review the solution?](https://reader033.vdocuments.site/reader033/viewer/2022052509/55a0d35b1a28ab59108b47ef/html5/thumbnails/30.jpg)
SAPO Websecurity Team
Open-Source
• Likely found by code review
• “A test case could have caught this, but it's difficult because it's so deep into the handshake.”
• “Code review can be effec9ve against these sorts of bug.”
17
![Page 31: Is code review the solution?](https://reader033.vdocuments.site/reader033/viewer/2022052509/55a0d35b1a28ab59108b47ef/html5/thumbnails/31.jpg)
SAPO Websecurity Team
Open-Source
• In 2011, a Ph.D student pushed a commit to OpenSSL that implemented the Heartbeat extension
• Reviewed by one of OpenSSL’s four core developers • code in C • the problem was not detected
18
![Page 32: Is code review the solution?](https://reader033.vdocuments.site/reader033/viewer/2022052509/55a0d35b1a28ab59108b47ef/html5/thumbnails/32.jpg)
SAPO Websecurity Team
Open-Source
19
![Page 33: Is code review the solution?](https://reader033.vdocuments.site/reader033/viewer/2022052509/55a0d35b1a28ab59108b47ef/html5/thumbnails/33.jpg)
SAPO Websecurity Team
Open-Source
19
![Page 34: Is code review the solution?](https://reader033.vdocuments.site/reader033/viewer/2022052509/55a0d35b1a28ab59108b47ef/html5/thumbnails/34.jpg)
SAPO Websecurity Team
Open-Source
19
![Page 35: Is code review the solution?](https://reader033.vdocuments.site/reader033/viewer/2022052509/55a0d35b1a28ab59108b47ef/html5/thumbnails/35.jpg)
SAPO Websecurity Team
Open-Source
19
![Page 36: Is code review the solution?](https://reader033.vdocuments.site/reader033/viewer/2022052509/55a0d35b1a28ab59108b47ef/html5/thumbnails/36.jpg)
SAPO Websecurity Team
Open-Source
19
![Page 37: Is code review the solution?](https://reader033.vdocuments.site/reader033/viewer/2022052509/55a0d35b1a28ab59108b47ef/html5/thumbnails/37.jpg)
SAPO Websecurity Team
Open-Source
20
![Page 38: Is code review the solution?](https://reader033.vdocuments.site/reader033/viewer/2022052509/55a0d35b1a28ab59108b47ef/html5/thumbnails/38.jpg)
SAPO Websecurity Team
Open-Source
20
![Page 39: Is code review the solution?](https://reader033.vdocuments.site/reader033/viewer/2022052509/55a0d35b1a28ab59108b47ef/html5/thumbnails/39.jpg)
SAPO Websecurity Team
Open-Source
• Heartbleed
• CVE-‐2014-‐0160 -‐ Allows reading of random data from the process memory
• Affected OpenSSL -‐ used by many exposed services such as www and mail
• hcp://pi5.5l.sl.pt
21
![Page 40: Is code review the solution?](https://reader033.vdocuments.site/reader033/viewer/2022052509/55a0d35b1a28ab59108b47ef/html5/thumbnails/40.jpg)
SAPO Websecurity Team
Open-Source
• Should have been detected with code review
• hcp://pi5.fp.sl.pt
22
![Page 41: Is code review the solution?](https://reader033.vdocuments.site/reader033/viewer/2022052509/55a0d35b1a28ab59108b47ef/html5/thumbnails/41.jpg)
SAPO Websecurity Team
Open-Source
• SQL injec9on
23
![Page 42: Is code review the solution?](https://reader033.vdocuments.site/reader033/viewer/2022052509/55a0d35b1a28ab59108b47ef/html5/thumbnails/42.jpg)
SAPO Websecurity Team
Open-Source
• SQL injec9on
23
![Page 43: Is code review the solution?](https://reader033.vdocuments.site/reader033/viewer/2022052509/55a0d35b1a28ab59108b47ef/html5/thumbnails/43.jpg)
SAPO Websecurity Team
Open-Source
• SQL injec9on
23
• hcp://vuln.example/login?username=x’ or 1=1 limit 0,1-‐-‐%20
• SELECT id,group,full_name FROM users WHERE username=’x’ or 1=1 limit 0,1-‐-‐
![Page 44: Is code review the solution?](https://reader033.vdocuments.site/reader033/viewer/2022052509/55a0d35b1a28ab59108b47ef/html5/thumbnails/44.jpg)
SAPO Websecurity Team
How to
• Code review methods vary a lot • highly dependent on the depth of the analysis
• Broad categories with different names depending on the author • Formal code review • Lightweight code review
24
![Page 45: Is code review the solution?](https://reader033.vdocuments.site/reader033/viewer/2022052509/55a0d35b1a28ab59108b47ef/html5/thumbnails/45.jpg)
SAPO Websecurity Team
How to
• Formal code review • line by line • mul9ple reviewers • group review • printed copies
• Finds hard to find problems • Time consuming
25
![Page 46: Is code review the solution?](https://reader033.vdocuments.site/reader033/viewer/2022052509/55a0d35b1a28ab59108b47ef/html5/thumbnails/46.jpg)
SAPO Websecurity Team
How to
• Lightweight code review • shallow analysis • pacern based analysis • grep based
• reviewing only cri9cal func9ons
• Prone do miss some problems • Less 9me consuming • Good to easily find certain classes of vulnerabili9es
26
![Page 47: Is code review the solution?](https://reader033.vdocuments.site/reader033/viewer/2022052509/55a0d35b1a28ab59108b47ef/html5/thumbnails/47.jpg)
SAPO Websecurity Team
How to
• Review can be done • manually • automa9cally • using both approaches
• Using both approaches • automa9cally find hotspots with pacern matching • manually review those areas
27
![Page 48: Is code review the solution?](https://reader033.vdocuments.site/reader033/viewer/2022052509/55a0d35b1a28ab59108b47ef/html5/thumbnails/48.jpg)
SAPO Websecurity Team
How to
• Combining approaches • milestone • mandatory review and approval before going to produc9on
• a posteriori • detec9on vs preven9on
• sampling • review just some code, chosen by • keyword • commiter • project
28
![Page 49: Is code review the solution?](https://reader033.vdocuments.site/reader033/viewer/2022052509/55a0d35b1a28ab59108b47ef/html5/thumbnails/49.jpg)
SAPO Websecurity Team
How to
• Basic rules for code review to work
• 1st rule: the reviewer must not be the one who wrote the code • if we could find bugs in our code we would be able to avoid them
• biased analysis • the reviewer will have a different and unbiased perspec9ve
• the reviewer should be from a different project
29
![Page 50: Is code review the solution?](https://reader033.vdocuments.site/reader033/viewer/2022052509/55a0d35b1a28ab59108b47ef/html5/thumbnails/50.jpg)
SAPO Websecurity Team
How to
• 2nd rule: the reviewer should understand the language being reviewed
30
![Page 51: Is code review the solution?](https://reader033.vdocuments.site/reader033/viewer/2022052509/55a0d35b1a28ab59108b47ef/html5/thumbnails/51.jpg)
SAPO Websecurity Team
How to
• 3rd rule: focus on the objec9ve: security, performance, feature, etc., but not on everything
31
![Page 52: Is code review the solution?](https://reader033.vdocuments.site/reader033/viewer/2022052509/55a0d35b1a28ab59108b47ef/html5/thumbnails/52.jpg)
SAPO Websecurity Team
More motivation
• How to mo9vate the reviewers?
32
![Page 53: Is code review the solution?](https://reader033.vdocuments.site/reader033/viewer/2022052509/55a0d35b1a28ab59108b47ef/html5/thumbnails/53.jpg)
SAPO Websecurity Team
More motivation
• Just saying “you must do code review” will not work • developers have more interes9ng stuff to do • developers have more stuff to do • developers have deadlines and code review is easily not taken in considera9on (1st to drop)
• developers don’t like others code • what to review?
33
![Page 54: Is code review the solution?](https://reader033.vdocuments.site/reader033/viewer/2022052509/55a0d35b1a28ab59108b47ef/html5/thumbnails/54.jpg)
SAPO Websecurity Team
How to
• What to review is a big ques9on • don’t let the developer choose what to review arbitrarily
• Assign “reviews” to reviewers • use a tool to manage what is assigned to each reviewer
• each reviewer has a queue of reviews to be done
34
![Page 55: Is code review the solution?](https://reader033.vdocuments.site/reader033/viewer/2022052509/55a0d35b1a28ab59108b47ef/html5/thumbnails/55.jpg)
SAPO Websecurity Team
How to
• Assign “reviews” to reviewers • for instance, single commits
• Ensures • coverage -‐ all code is reviewed • responsibility -‐ the developer has something publicly assigned to him
• deliverables -‐ audit evidence; increases mo9va9on to review
35
![Page 56: Is code review the solution?](https://reader033.vdocuments.site/reader033/viewer/2022052509/55a0d35b1a28ab59108b47ef/html5/thumbnails/56.jpg)
SAPO Websecurity Team
How to
• Even with task assignment the reviewer might let the work pile up • its like documenta9on: the applica9on will work fine without it
• This will happen if the review is done individually and on their usual sirng place • gather developers
36
![Page 57: Is code review the solution?](https://reader033.vdocuments.site/reader033/viewer/2022052509/55a0d35b1a28ab59108b47ef/html5/thumbnails/57.jpg)
SAPO Websecurity Team
How to
• Book a mee9ng room • Get the developers there
37
![Page 58: Is code review the solution?](https://reader033.vdocuments.site/reader033/viewer/2022052509/55a0d35b1a28ab59108b47ef/html5/thumbnails/58.jpg)
SAPO Websecurity Team
Tools
• Suppor9ng sobware
• Phabricator • repository integra9on • assignment rules • issue tracking • pre and post commit hooking
38
![Page 59: Is code review the solution?](https://reader033.vdocuments.site/reader033/viewer/2022052509/55a0d35b1a28ab59108b47ef/html5/thumbnails/59.jpg)
SAPO Websecurity Team
Tools
• Phabricator
39
![Page 60: Is code review the solution?](https://reader033.vdocuments.site/reader033/viewer/2022052509/55a0d35b1a28ab59108b47ef/html5/thumbnails/60.jpg)
SAPO Websecurity Team
Tools
• Phabricator
40
![Page 61: Is code review the solution?](https://reader033.vdocuments.site/reader033/viewer/2022052509/55a0d35b1a28ab59108b47ef/html5/thumbnails/61.jpg)
SAPO Websecurity Team
Tools
• Gerrit
41
![Page 62: Is code review the solution?](https://reader033.vdocuments.site/reader033/viewer/2022052509/55a0d35b1a28ab59108b47ef/html5/thumbnails/62.jpg)
SAPO Websecurity Team
Tools
• Gerrit • pre-‐commit only • Git only
• Phabricator • pre-‐commit • post-‐commit • Subversion, Git, Mercurial
42
![Page 63: Is code review the solution?](https://reader033.vdocuments.site/reader033/viewer/2022052509/55a0d35b1a28ab59108b47ef/html5/thumbnails/63.jpg)
SAPO Websecurity Team
Tools
• Security Lib – less code to review
43
![Page 64: Is code review the solution?](https://reader033.vdocuments.site/reader033/viewer/2022052509/55a0d35b1a28ab59108b47ef/html5/thumbnails/64.jpg)
SAPO Websecurity Team
Tools
• Watch Commits
44
![Page 65: Is code review the solution?](https://reader033.vdocuments.site/reader033/viewer/2022052509/55a0d35b1a28ab59108b47ef/html5/thumbnails/65.jpg)
SAPO Websecurity Team
Tools
• Do not confuse code review with other mechanisms • sta9c analysis • dynamic analysis
• These lack human intelligence • but do not get 9red
45
![Page 66: Is code review the solution?](https://reader033.vdocuments.site/reader033/viewer/2022052509/55a0d35b1a28ab59108b47ef/html5/thumbnails/66.jpg)
SAPO Websecurity Team
Problems
• A portuguese company working in mission-‐cri9cal systems used (uses?) the following approach • developers get a printed A4 page with code • they also get a 5/6 items checklist • 15 min mee9ng in the next morning to discuss the checklist issues
• repeat every day
• Scrum alike methodology
46
![Page 67: Is code review the solution?](https://reader033.vdocuments.site/reader033/viewer/2022052509/55a0d35b1a28ab59108b47ef/html5/thumbnails/67.jpg)
SAPO Websecurity Team
Problems
• Problems with this approach?
47
• Feels like homework • might review at work but subject to the usual constraints
• Context • calls to func9ons outside the printed code • classes/objects defined elsewhere • inclusion of files and configura9ons
![Page 68: Is code review the solution?](https://reader033.vdocuments.site/reader033/viewer/2022052509/55a0d35b1a28ab59108b47ef/html5/thumbnails/68.jpg)
SAPO Websecurity Team
Problems
48
![Page 69: Is code review the solution?](https://reader033.vdocuments.site/reader033/viewer/2022052509/55a0d35b1a28ab59108b47ef/html5/thumbnails/69.jpg)
SAPO Websecurity Team
Problems
• Limita9ons • variables, objects and func9ons define outside • configura9on dependent execu9on • scope limita9on
49
![Page 70: Is code review the solution?](https://reader033.vdocuments.site/reader033/viewer/2022052509/55a0d35b1a28ab59108b47ef/html5/thumbnails/70.jpg)
SAPO Websecurity Team
Is code review the solution?
• Is code review the solu9on?
50
![Page 71: Is code review the solution?](https://reader033.vdocuments.site/reader033/viewer/2022052509/55a0d35b1a28ab59108b47ef/html5/thumbnails/71.jpg)
SAPO Websecurity Team
Is code review the solution?
• Is code review the solu9on?
50
• No.
• But it is a good complement • detects vulnerabili9es hard to find using blackbox approaches
• detects potencial problems, before they are exploitable
![Page 72: Is code review the solution?](https://reader033.vdocuments.site/reader033/viewer/2022052509/55a0d35b1a28ab59108b47ef/html5/thumbnails/72.jpg)
SAPO Websecurity Team
More
• Other presenta9ons
– slideshare.net/9agomendo – slideshare.net/nuno.loureiro –AP2SI -‐ facebook.com/ap2si –OWASP -‐ owasp.org
51