slide used at acm-sac 2014 by suzaki
DESCRIPTION
Slide used at ACM-SAC (Symposium On Applied Computing) 2014. The title is "Rollback Mechanism of Nested Virtual Machines for Protocol Fuzz Testing".TRANSCRIPT
National Institute of Advanced Industrial Science and Technology
Rollback MechanismRollback Mechanism of Nested Virtual Machines for Protocol Fuzz Testing
Kuniyasu Suzaki*, Toshiki Yagi*, Akira Tanaka*, Y taka Oi a* Ets a Shiba ama*‡Yutaka Oiwa , Etsuya Shibayama ‡
* National Institute of Advanced Industrial Science and Technology(AIST) National Institute of Advanced Industrial Science and Technology(AIST) ‡The University of Tokyo
ACM SAC(Symposium On Applied Computing)2014 at Gyeongju, South Korea March 25
National Institute of Advanced Industrial Science and Technology
Motivation 1/2Motivation 1/2• Sec re protocols (SSL/TLS SSH etc) are important for• Secure protocols (SSL/TLS, SSH, etc) are important for
Internet commerce.l bili i f i l i f• However, vulnerabilities of implementation of secure
protocols are discovered repeatedly. – Incidents of SSL/TLS vulnerabilities
• Insufficient entropy of random numbers [2012] – It creates same secret-keys for SSL/TLS certificates.It creates same secret keys for SSL/TLS certificates.– EFF offers a site to check SSL certificates.
» https://www.eff.org/observatory
• CRIME[2012]• CRIME[2012]– It attacks to the data compression used by the TLS and SPDY.
• Lucky Thirteen[2013]– It attacks padding oracle using timing side-channel.
National Institute of Advanced Industrial Science and Technology
Motivation 2/2Motivation 2/2• The implementations of secure protocols should be verified.
• Our group increases TLS security with 2 approaches.F l ifi i– Formal verification
• Verify important parts of protocol hander program.– “Formal Network Packet Processing with Minimal Fuss: Invertible Syntax Descriptions atFormal Network Packet Processing with Minimal Fuss: Invertible Syntax Descriptions at
Work.” [PLPV’12]– “Towards Formal Verification of TLS Network Packet Processing Written in C”
[PLPV’13]
– Exhaustive Protocol Fuzz Testing• Try to check all reaction of protocol handler.
– Protocol Test Generator creates and sends many fuzzed packets to check implementation of protocol handler.
– (Today’s Topic) Computing environment for protocol fuzz testing
National Institute of Advanced Industrial Science and Technology
O liOutlineM i i• Motivation
• Protocol Test Generator: fuzz testing on a protocol handler• Why we use nested VM for protocol fuzz testing?• Design issuesDesign issues• Implementation
P f l i• Performance evaluation on current prototype.• Conclusion
National Institute of Advanced Industrial Science and Technology
Protocol Test GeneratorV if t l h d f h t l t t• Verify a protocol hander for each protocol state– Repeat the following sequence for each protocol state
1. Creates a fuzzed packet2. Send it to a protocol hander 3. Check the reply
• Fuzzed packets are tested on each protocol stateFuzzed packets are tested on each protocol state many times.
• Snapshot and rollback mechanism is required.S p q
National Institute of Advanced Industrial Science and Technology
Sequence of Protocol Test Generator (TLS)Sequence of Protocol Test Generator (TLS)Normal
TargetProtocol Generator
Client Hello
Take Snapshot
Client Hello
Server Hello
Server Certificate
Snapshot1
Server Hello Done
Take SnapshotTake Snapshot
Client Key Exchange… Snapshot 2
National Institute of Advanced Industrial Science and Technology
Sequence of Protocol Test Generator (TLS)Sequence of Protocol Test Generator (TLS)Normal Fuzz for Client Hello
TargetProtocol Generator TargetProtocol Generator
Client Hello
Take Snapshot
[Fuzz1] Client Hello
Rollback Snapshot1
Client Hello
Server Hello
Server Certificate
Snapshot1 [Fuzz1] Client Hello
????
Good or Bad?
Rollback1
Server Hello Done
Take Snapshot Rollback Snapshot1
Good or Bad?
Take Snapshot
Client Key Exchange… Snapshot 2
[Fuzz2] Client Hello
????Rollback 1
Good or Bad?
[FuzzN] Client Hello
National Institute of Advanced Industrial Science and Technology
Sequence of Protocol Test Generator (TLS)Sequence of Protocol Test Generator (TLS)Normal Fuzz for Client Hello Fuzz for Client Key Exchange
TargetProtocol Generator TargetProtocol Generator TargetProtocol Generator
Client Hello
Take Snapshot
[Fuzz1] Client Hello
Rollback Snapshot1[Fuzz1] Client Key
Rollback Snapshot2
Client Hello
Server Hello
Server Certificate
[Fuzz1] Client Hello
????
Good or Bad?
Snapshot1 Rollback1
[Fuzz1] Client Key Exchange
???
Rollback2
Server Hello Done
Take Snapshot Rollback Snapshot1
Good or Bad?
Rollback Snapshot2
Good or Bad?
Take Snapshot
Client Key Exchange…
[Fuzz2] Client Hello
????Snapshot 2 Rollback 1 Rollback 2
[Fuzz2] Client Key Exchange
???
[FuzzN] Client Hello
Good or Bad????
Good or Bad?[FuzzN] Client Key Exchange
National Institute of Advanced Industrial Science and Technology
Requirement for Protocol Test GeneratorRequirement for Protocol Test Generator1. Packet level granularity control
– Generator wants to take a snapshot just before sending a packet.
2. Hardware-level repeatabilityB l biliti d b it ti i t (– Because some vulnerabilities are caused by its computing environment (e.g., random number)
3. Consistency between generator and target protocol hander3. Consistency between generator and target protocol hander– Most snapshot/rollback mechanism does not care of packets on a wire.
1. We propose a special protocol to control target computing environment.
2. We uses Virtual Machine which offers hardware-level repeatability f h / llb kfor snapshot/rollback.
3. Design issues. (Mention in later slides)
National Institute of Advanced Industrial Science and Technology
Proposed Protocol (VTP: Virtual Test Protocol)Proposed Protocol (VTP: Virtual Test Protocol)• VTP treats packets of test-target protocol as data.
T t t t k t l t d t “C l ” k t– Test-target packets are en-capsulated to “Capsule” packets– “Capsule” packets must be de-capsulated on target computing environment.
• VTP has special packets to control target computing environment• VTP has special packets to control target computing environment– “Take_Snapshot” and “Rollback”
Protocol Action
CapsulePackets of secure communication are encapsulated with capsule protocol. The flow of capsulated packets is managed
h t k h t llb k k t i i dwhen take_snapshot or rollback packet is issued. Take_ Snapshot
Take VM snapshot image. Packets on the wire must be managed by a certain method. Return snapshot ID.R h i C i fRollback
(with ID)
Resume snapshot image. Connection of secure communication must be operative, when a previous image is resumed.
Target computing environment (VM) has to deal with the VTP.
National Institute of Advanced Industrial Science and Technology
Design Issues for VM (Computing environment)Design Issues for VM (Computing environment)
• Virtual Machine runs a protocol handler.• En-capsulated packets by VTP must be de-capsulated on a target VM.
– Original packets must be passed to a target protocol handler on the VM.
• The control packets (take_snapshot and rollback) must be dealt by the VM.• The VTP packets on the network must be maintained.
– We have to deal with consistent global state defied by [Chandy-Lamport, 1985].
ProtocolGenerator
Target EnvironmentSnapshots of VM
VTP
Protocol handler
TLS
VTP
VM deals with VTP protocol. Packets on the network must be maintained.
VM
National Institute of Advanced Industrial Science and Technology
Design choice for handling VTPDesign choice for handling VTP • Customize VM’s network function for VTPCustomize VM s network function for VTP
– We planned to customize SLiRP of QEMU• However VM control (snapshot/rollback) from SLiRP is strange, because it ( p ) g ,
is a part of VM. It is not flexible.
• Nested VM and Proxies (external & internal)– External proxy manages VTP control packets and takes snapshot of
External VM. – A snapshot of External VM keeps connectivity of network between
internal proxy and a protocol handler on Internal VM.• Pros: Easy and flexible implementation.
» It utilizes existing VM control commands. Don’t need to change VM.• Cons: Overhead? Performance is improved by many previous researchesCons: Overhead? Performance is improved by many previous researches.
National Institute of Advanced Industrial Science and Technology
Design: Nested VM and Proxiesg• External VM (a unit of snapshot)
– It is used to take snapshot of Internal VM and Internal Proxy.• Internal VM (hardware repeatability)( p y)
– It includes protocol hander and keeps connection with Internal Proxy, when External VM is taken a snapshot.• External Proxy
– Connects to Protocol Generator directly, and controls snapshot/rollback of External VM.
Real Machine External VMControl the target VM
• Internal Proxy– Maintain connectivity of VTP Protocol when a snapshot image is resumed.
Real Machine External VM
Internal VM
Control the target VM(SAVEVM, LOADVM)
Decapsulatecontrol commands
InternalProxy openSSL
Port forward
ExternalProxyExternalProxy
InternalProxyTLS
VTP
TLSTLS
VTP
External VM
Internal VM
External VM
Internal VM
ProtocolGenerator “take_snapshot” packet make a
round‐trip between external and internal proxy to confirm no VTP packets on the wire
13
Internal VMSSH
openSSLKeep connectionSnapshots
no VTP packets on the wire.
RollbackTLS
National Institute of Advanced Industrial Science and Technology
Handling VTP “capsule” packetHandling VTP capsule packet• Protocol Generator en-casuplates test-target packets with “capsule” packets. • External Proxy passes capsule packets to Internal Proxy without change.• Internal Proxy de-capsulates VTP packets and passes original packets to protocol
hander on Internal VMhander on Internal VM.
Real Machine External VMReal Machine External VM
Internal VMDecapsulatecontrol commands
InternalProxy openSSL
Port forward
ExternalProxyExternalProxy
InternalProxyTLS
VTP
TLSTLS
VTP
ProtocolGenerator
National Institute of Advanced Industrial Science and Technology
Handling VTP “take snapshot” packetHandling VTP take_snapshot packet• When a “take_snapshot” packet is issued, the packet makes a round trip between
External and Internal Proxy to confirm that there is no VTP packet on the wire (consistent global state defied by Chandy Lamport)(consistent global state defied by Chandy-Lamport).
• After that, External Proxy sends SAVEM command to External VM to take a snapshot.
• Connection between Internal Proxy and protocol handler is kept in a snapshot.
Real Machine External VMControl the target VMReal Machine External VM
Internal VM
g(SAVEVM)
Decapsulatecontrol commands
InternalProxy openSSL
Port forward
ExternalProxyExternalProxy
InternalProxy
P t l
TLS
VTP
TLSTLS
VTP
External VM
Internal VM
External VM
Internal VM
ProtocolGenerator “take_snapshot” packet make a
round‐trip between external and internal proxy to confirm no VTP packets on the wire.
15
Internal VMSSH
openSSLKeep connectionSnapshots
no VTP packets on the wire.TLS
National Institute of Advanced Industrial Science and Technology
Handling VTP “rollback” packetHandling VTP rollback packet• When a “rollback” packet is issued, the External Proxy send
LOADVM command to External VM It resumes an old imageLOADVM command to External VM. It resumes an old image.• After that, External and Internal proxies re-establish the connection
between them, because they loses the network states when rollback. , y– Internal proxy and protocol handler do not need to care about network connection.
Real Machine External VMControl the target VMReal Machine External VM
Internal VM
g(LOADVM)
Decapsulatecontrol commands
InternalProxy openSSL
Port forward
ExternalProxyExternalProxy
InternalProxy
P t l
TLS
VTP
TLSReestablish
External VM
Internal VM
External VM
Internal VM
ProtocolGenerator
16
Internal VMSSH
openSSLKeep connection Rollback
TLS
National Institute of Advanced Industrial Science and Technology
OptimizationOptimization• Most overhead of VM is caused by video emulation.
– Nested VMs must emulate the GUI of internal VM on external VM again.g
• Fortunately QEMU and KVM have “curses mode” which emulates text user interface.
• We eliminate GUI on both internal and external VMs.N GUI d d k h t i k– No GUI reduces memory usage and make snapshot quick.
GUIs of Nested VMs No GUI of Nested VMs
OpenSSL on Linux onInternal VM (QEMU)
on External VM (KVM)ith d
Internal VM (QEMU)
OpenSSL on Windows
Extenal Proxy
Protocol Tester
with curses mode
Protocol Tester
External Proxy
External VM (KVM)Internal Proxy on
Linux
National Institute of Advanced Industrial Science and Technology
Current ImplementationCurrent Implementation• We implement Nested VM and proxies on normalWe implement Nested VM and proxies on normal
Debian GNU Linux x86. – External VM: KVMExternal VM: KVM
• The VM is a unit of snapshot and is NOT required hardware repeatability.
• KVM is fast with virtualization technology of X86.
– Internal VM: QEMU h l h d f bili ( d b• The VM emulates hardware for repeatability (e.g., random number
generator).
– Proxies are written by PerlProxies are written by Perl.• External Proxy: 430 LOC • Internal Proxy: 132 LOC• Internal Proxy: 132 LOC
National Institute of Advanced Industrial Science and Technology
Performance evaluationPerformance evaluation • We evaluated performance on Protocol Test Generator.
– Test for hand shake of TLS (PolarSSL) takes 9 snapshots and 2,311 rollbacks.– Effect of GUI and noGUI.– Effct of Memory size (large 1024MB/512MB small 512MB/256MB)– Effct of Memory size (large 1024MB/512MB, small 512MB/256MB)
GUIMem:
NoGUIMem:
NoGUIMem:
1024/512(sec)
1024/512(sec)
512/256(sec)
Setting up nested VMs 266 107 93g pFuzz packet generator 1,307 1,164 1,080
Nested VMs and Proxies
Snapshot(9 times)
57(Av 6.33)
37(Av 4.11)
24(Av 2.67)
R llb k 3 135 2 197 1 286 VM overhead isRollback(2,311times)
3,135(Av 1.36)
2,197(Av 0.96)
1,286(Av 0.56)
Other 12 13 12Total 5 043 3 622 2 587
VM overhead is almost same to packet generator.
Total 5,043 3,622 2,587
ThinkPad T410 (CPU Intel Core i7-M620 2.67Ghz, Memory 4GB).
National Institute of Advanced Industrial Science and Technology
R lt f P t l F T tiResults of Protocol Fuzz Testing W ifi d th h d h k f 4 TLS• We verified the handshake of 4 TLS servers.– OpenSSL, GnuTLS, CyaSSL, and PolarSSL.
• The fuzz testing found 2 bugs in CyaSSL– 2 bugs in CyaSSL
– 1 bug in PolarSSL• We reported the results to the mailing list, and the
bugs were fixed.g
National Institute of Advanced Industrial Science and Technology
R l t d kRelated works• Nested VM• Nested VM
– Migration• Xen-Blanket [EuroSys’12] enables user level migration on CloudXen Blanket [EuroSys 12] enables user level migration on Cloud.
– Security• CloudVisor [SOSP’11] is inserted on existing VM on multi-tenant
Cloud.
• Process level snapshot– DMTCP: Distributed Multi Threaded Check Pointing
• http://dmtcp.sourceforge.net/If d ’t t d t b t h d l l t bilit f• If we don’t not need to care about hardware-level repeatability for rollback, DMTCP will be a candidate for protocol test generator.
National Institute of Advanced Industrial Science and Technology
Conclusion• We propose a combination of nested VM and proxies
for protocol fuzz testing environment.– It treats hardware-level repeatability (e.g., random number).– It is easy and flexible to implement.
• Our method is not limited to protocol testing. It can be applied on other areas (e.g., I/O fuzz testing, memory pp ( g , g, yfuzz testing).
AcknowledgementThis work is supported by the National Institute of Information and Communications Technology of Japan.