Upload File

neuroninspect: detecting trojan backdoors in deep neural … · 2020. 4. 18. · trojan backdoor...

  • Home
  • Documents
  • NeuronInspect: Detecting Trojan Backdoors in Deep Neural … · 2020. 4. 18. · Trojan Backdoor Attack Deep neural network has achieved state-of-the-art performance on various tasks
1
Trojan Backdoor Attack Deep neural network has achieved state-of-the-art performance on various tasks. However, lack of interpretability and transparency makes it easier for malicious attackers to inject trojan backdoor into the deep neural networks, which will make the model behave abnormally. Our goal is to identify whether a given deep neural network contains a malicious trojan backdoor. NeuronInspect: Detecting Trojan Backdoors in Deep Neural Networks via Visual Interpretability Xijie Huang 1 , Moustafa Alzantot 2 , Mani Srivastava 2 1 Shanghai Jiao Tong University 2 University of California, Los Angeles Introduction Algorithm Design Experiments Conclusion Visual Interpretability Using only a set of few clean examples, we use visual interpretability technique to generate a saliency map for each labels. Across different images, we notice that for backdoored DNNs the explanation of the target label will look significantly different from other labels in terms of being: sparse, centralized and unique. Experiment Setup Multiple triggers Ablation Studies GTSRB Traffic Sign Recognition Dataset [2] MNIST Digit Recognition Dataset we proposed NeuronInspect, the first approach effectively detect the trojan backdoor in DNNs without backdoor samples or restoring the trigger. We extensively evaluate it on various attack scenarios and prove better robustness and effectiveness over state-of-the-art backdoor detection techniques Neural Cleanse [3] by a great margin. Smoothness Sparseness Uniqueness Combine Feature Features -> Outlier Detection Efficiency [1] Gu, Tianyu, et al. "Badnets: Identifying vulnerabilities in the machine learning model supply chain." [2] Stallkamp, Johannes, et al. "Man vs. computer: Benchmarking machine learning algorithms for traffic sign recognition." [3] Wang, Bolun, et al. "Neural Cleanse: Identifying and Mitigating Backdoor Attacks in Neural Networks." We attack classification DNNs on various dataset with different trigger pattern, size and location following configurations of BadNets [1]. MNIST GTSRB T-Thresholding XOR

Upload: others

Post on 26-Aug-2020

1 views

Category:

Documents


0 download

Report

TRANSCRIPT

Page 1: NeuronInspect: Detecting Trojan Backdoors in Deep Neural … · 2020. 4. 18. · Trojan Backdoor Attack Deep neural network has achieved state-of-the-art performance on various tasks

Trojan Backdoor Attack Deep neural network has achieved state-of-the-art performance on various tasks. However, lack of interpretability and transparency makes it easier for malicious attackers to inject trojan backdoor into the deep neural networks, which will make the model behave abnormally. Our goal is to identify whether a given deep neural network contains a malicious trojan backdoor.

NeuronInspect: Detecting Trojan Backdoors in Deep Neural Networks via Visual Interpretability

Xijie Huang1, Moustafa Alzantot2, Mani Srivastava2

1Shanghai Jiao Tong University 2University of California, Los Angeles

Introduction

Algorithm Design

Experiments

Conclusion

Visual Interpretability Using only a set of few clean examples, we use visual interpretability technique to generate a saliency map for each labels. Across different images, we notice that for backdoored DNNs the explanation of the target label will look significantly different from other labels in terms of being: sparse, centralized and unique.

Experiment Setup

Multiple triggers

Ablation Studies

GTSRB Traffic Sign Recognition Dataset [2]

MNIST Digit Recognition Dataset

we proposed NeuronInspect, the first approach effectively detect the trojan backdoor in DNNs without backdoor samples or restoring the trigger. We extensively evaluate it on various attack scenarios and prove better robustness and effectiveness over state-of-the-art backdoor detection techniques Neural Cleanse [3] by a great margin.

Smoothness

Sparseness

Uniqueness

Combine FeatureFeatures -> Outlier Detection

Efficiency

[1] Gu, Tianyu, et al. "Badnets: Identifying vulnerabilities in the machine learning model supply chain."[2] Stallkamp, Johannes, et al. "Man vs. computer: Benchmarking machine learning algorithms for traffic sign recognition."[3] Wang, Bolun, et al. "Neural Cleanse: Identifying and Mitigating Backdoor Attacks in Neural Networks."

We attack classification DNNs on various dataset with different trigger pattern, size and location following configurations of BadNets [1].

MNIST

GTSRB

T-Thresholding XOR

Neural Networks - cs.cmu.edubhiksha/courses/deep... · about neural networks (a.k.a deep learning) This guy learned about neural networks (a.k.a deep learning) Objectives of this

Deep Neural Networks for Object Detectionpapers.nips.cc/.../5207-deep-neural-networks-for-object-detection.pdf · Deep Neural Networks for Object Detection ... recognition becomes

Do Deep Neural Networks Suffer from Crowding?papers.nips.cc/paper/...deep-neural-networks-suffer-from-crowding.pdf · Do Deep Neural Networks Suffer from Crowding? Anna Volokitiny\

Artificial Neural Networks and Deep Learningchrome.ws.dei.polimi.it/images/a/ae/AN2DL_02_2021_Prceptron_2_Fee… · Artificial Neural Networks and Deep Learning . 2 «Deep Learning

ImageNet Classification with Deep Convolutional Neural ...rogerioferis.com/.../presentations/GuangnanAndMajaDeepLearning.pdf · ImageNet Classification with Deep Convolutional Neural

Designing Trojan Detectors in Neural Networks Using

ImageNet Classification with Deep Convolutional Neural …kriz/imagenet_classification_with_deep_convolutional.pdfImageNet Classification with Deep Convolutional Neural Networks

Deep Neural Decision Forests

Introduction to Deep Neural Networks - Deep Learning · Introduction to Deep Neural Networks 0. Logistics Spring 2020 1. Neural Networks are taking over! •Neural networks have become

Introduction to Deep Neural Networksbhiksha/courses/deep... · Introduction to Deep Neural Networks 0. Logistics Spring 2020 1. Neural Networks are taking over! •Neural networks

Deep Neural Networks in Machine Translation: An …perpustakaan.unitomo.ac.id/repository/Deep Neural...Deep Neural Networks in Machine Translation: An Overview Jiajun Zhang and Chengqing

Parallelized Deep Neural Networks

Distributed Deep Neural Networks over the Cloud, the …system has built-in support for automatic ... Keywords-distributed deep neural networks; deep neural net ... the network to

Deep Q-Learning with Recurrent Neural Networkscs229.stanford.edu/...DeepQLearningWithRecurrentNeuralNetwords... · Deep Q-Learning with Recurrent Neural Networks ... rent neural network

ImageNet Classification with Deep Deep Convolutional …rita/visual_recog_course/talks/ImageNet... · ImageNet Classification with Deep Deep Convolutional Convolutional Neural Neural

Troubleshooting Deep Neural Networks - josh-tobin.comjosh-tobin.com/assets/pdf/troubleshooting-deep-neural-networks-01... · Josh Tobin. January 2019. josh-tobin.com/troubleshooting-deep-neural-networks

STRIP: A Defence Against Trojan Attacks on Deep Neural ... · Detecting Trojan Attack is challenging 16 Trojantrigger can be at any shape, sizeand pattern Freely chosenby attackers

Optimizing Deep Neural Networks

Deep Neural Networks - dmi.unibas.ch

Trojan Battery T-105 Plus Deep Cycle For Golf Cars … Trojan Battery T-105 Plus Deep Cycle For Golf Cars Solar Energy and Back Up Battery Author Trojan Battery Subject 6 Volt Deep

Practical Deep Neural Networks - DGYrt.dgyblog.com/res/dlworkshop/introduction.pdf · Practical Deep Neural Networks ... ' CS231n Convolutional Neural Networks for Visual Recognition

Debugging deep neural networks

Lecture 10: Neural Networks and Deep Learningsaravanan-thirumuruganathan.github.io/cse5334... · Neural Networks Deep Learning Convolutional Neural Networks Recurrent Neural Networks

An Embarrassingly Simple Approach for Trojan Attack in Deep … · 2020-06-19 · An Embarrassingly Simple Approach for Trojan Attack in Deep Neural Networks Ruixiang Tang, Mengnan

Deep Neural Network Approach for the Dialog State Tracking ...people.cs.pitt.edu/~huynv/research/deep-nets/Deep Neural Network Approach for the...Deep Neural Network Approach for the

Cnvlutin: Ineffectual-Neuron-Free Deep Neural Network …enright/albericio-isca2016.pdf · 2016-08-02 · Cnvlutin: Ineffectual-Neuron-Free Deep Neural ... Deep Neural Networks (DNNs)

Design and Evaluation of a Multi-Domain Trojan Detection ... · 1 Design and Evaluation of a Multi-Domain Trojan Detection Method on Deep Neural Networks Yansong Gao, Yeonjae Kim,

Applied inductive learning - Lecture 5 (Deep) Neural Networkslwh/AIA/deep-neural-nets-29-12... · 2019-01-02 · Applied inductive learning - Lecture 5 (Deep) Neural Networks

Memory Trojan Attack on Neural Network Accelerators · 2020. 1. 1. · Memory Trojan Attack on Neural Network Accelerators Yang Zhao 1, Xing Hu , Shuangchen Li , Jing Ye2;3, Lei Deng

Deep Convolutional Neural Network for Image …papers.nips.cc/paper/5485-deep-convolutional-neural...Deep Convolutional Neural Network for Image Deconvolution Li Xu ∗ LenovoResearch

Deep Learning Binary Neural Network on an FPGA · Deep Learning Binary Neural Network on an ... deep neural networks have attracted lots of attentions ... brain has a deep architecture

Deep Learning in Neural Networks: An Overviejuergen/DeepLearning28May2014.pdf · Deep Learning in Neural Networks: An Overview ... 1 Introduction to Deep Learning ... Neural Fitted

Deep Neural Networks - uchile.cl

Special Topic - University of Georgiacobweb.cs.uga.edu/~khaled/MLcourse/Deep_Learning_Lecture.pdf · Deep Neural Networks Deep (but not that deep) Neural Network 6. Deep Neural Networks

STRIP: A Defence Against Trojan Attacks on Deep Neural … · 2020-01-20 · 1 STRIP: A Defence Against Trojan Attacks on Deep Neural Networks Yansong Gao, Chang Xu, Derui Wang, Shiping