security for machine learning systems · computer architectureand robust energy-efficient...

Computer Architecture and RobustEnergy-Efficient Technologies Group

CAREech

Security for Machine Learning Systems Attacks, Defenses, & Reconfigurability

M. Shafique

2

Who Ruled the World!2

Age of PowerMan-Power (#), Skills, Strength, Courage, etc.

Age of Resources and IndustryFuel, Industrial Tech., Economic Politics, etc.

Age of Data and AIData is the New Fuel

Innovation in Technology is the New PoliticsNation-wide Race for Dominance in AI

3

Smart Cyber Physical Systems & Internet-of-Things

Smart Traffic ControlSmart Transport Systems

Smart Gridshttp://solutions.3m.com/wps/portal/3M/en_EU/Sma

rtGrid/EU-Smart-Grid/

https://www.automotiveworld.com/analysis/automotive-cyber-physical-systems-next-computing-revolution/

https://www.emaze.com/@ACIOWOWR/IMSA-Slide-Show

Industry 4.0: Smart Industrial Automation

https://vimeo.com/145877805

Smart Houseshttps://www.linkedin.com/pulse/smart-homes-private-secure-future-intelligent-home-tripti-jha

Smart Automobileshttp://www.it5g.com/latest-software-enhancements-in-the-auto-industry/

Smart Robotshttp://alpha-smart.com/alphaboten

Smart Health Care

Applications

http://solutions.3m.com/wps/portal/3M/en_EU/SmartGrid/EU-Smart-Grid/https://www.automotiveworld.com/analysis/automotive-cyber-physical-systems-next-computing-revolution/https://www.emaze.com/@ACIOWOWR/IMSA-Slide-Showhttps://vimeo.com/145877805https://www.linkedin.com/pulse/smart-homes-private-secure-future-intelligent-home-tripti-jhahttp://www.it5g.com/latest-software-enhancements-in-the-auto-industry/http://alpha-smart.com/alphaboten

4

Smart CPS & IoT => The Big Data Processing Challenge!4

By 2025: 75 Billion connected devices By 2025: 4,785 interactions per

connected person in one day.

By 2025: 160 Zettabyte measured data By 2020: for every person on earth,

1.7 MB data/sec will be generated.

5

Smart CPS & IoT => The Big Data Processing Challenge!5

By 2025: 75 Billion connected devices By 2025: 4,785 interactions per

connected person in one day.

By 2025: 160 Zettabyte measured data By 2020: for every person on earth,

1.7 MB data/sec will be generated.

AI / ML is inevitable, we have to efficiently infer knowledge from the

big data, and derive predictions

6

ML Applications => requiring High Efficiency Gains6

Autonomous Driving

Cancer DetectionNatural Language

Processing

Image Classification

Object Detection & Localization

Machine Translation

Strategy Games

Forex/Stocks Trading

7

Machine Learning and the Computing Efficiency Gap!

Deep Blue vs. Garry Kasparov AlphaGO vs. Lee Sedol

IBM RS/6000 SP high-performance computer

1996 2016

IBM Watson vs. Brad Rutter and Ken Jennings

IBM Blue Gene supercomputer

1,920 CPUs and 280 GPUs

2011

Google's cloud computing

2,880 POWER7 processor threads and 16 terabytes of

RAM

Data Source: https://jacquesmattheij.com/another-way-of-looking-at-lee-sedol-vs-alphago

30 P2SC nodes + 480

dedicated ASICs to play

chess

1MW

200KW

20W900W + Power for 480 dedicated ASICs

8

Machine Learning and the Computing Efficiency Gap!

Deep Blue vs. Garry Kasparov AlphaGO vs. Lee Sedol

IBM RS/6000 SP high-performance computer

1996 2016

IBM Watson vs. Brad Rutter and Ken Jennings

IBM Blue Gene supercomputer

1,920 CPUs and 280 GPUs

2011

Google's cloud computing

2,880 POWER7 processor threads and 16 terabytes of

RAM

Data Source: https://jacquesmattheij.com/another-way-of-looking-at-lee-sedol-vs-alphago

30 P2SC nodes + 480

dedicated ASICs to play

chess

1MW

200KW

20W900W + Power for 480 dedicated ASICs

We need 10,000x Gains to Bridge this Gap!!!

9

Brain-Inspired Computing: [email protected]

Source: S. Mitra, P. Wong (Stanford), C. Mackin (MIT), J. Zhang (Google)

Software (Multi-Cores, GPUs)

Approximate Computing

Deep Learning Architectures

Neuromorphic Architectures and Beyond

Post-CMOS Technologies

......

... ... ... ...

Accurate SAD Accelerators

...

...

... ... ... ...

Approximate SAD Accel. Variant1

... ......

...

... ... ... ...

Approximate SAD Accel. VariantN-1

...

...

... ... ... ...

Approximate SAD Accel. VariantN

SAD Accelerator Array Output and Monitoring

(MV, SAD, etc.)

AGU

On-Chip Memory

Ref. FrameSearch Window

Current Frame CTU

Power-Gating Control Approximate Variant Selection Unit

User ConstraintsCPU Main Memory

System Bus

CPU executes the ME

algorithm andHEVC

Memory stores the video frames

1 uJ/neuron

FC layer

C1

1234567890

C2P1 P2 FC

Input Image

First ConvolutionC1@6 filters

PoolingP1@Average

Second ConvolutionC2@12 filters

PoolingP2@Average

Source: Huawei Kirin Chip

Source: IBM, TrueNorth Chip

??

In-Memory Computing

Source: IBM Research

CPS

High Performance, Energy Efficiency,

Configurability, Reliability & Security

*

...

...

.

.

.

.

.

.

.

.

.

.

.

.

Accurate SAD

Accelerators

...

...

.

.

.

.

.

.

.

.

.

.

.

.

Approximate SAD

Accel. Variant

1

.

.

.

.

.

.

...

...

.

.

.

.

.

.

.

.

.

.

.

.

Approximate SAD

Accel. Variant

N-1

...

...

.

.

.

.

.

.

.

.

.

.

.

.

Approximate SAD

Accel. Variant

N

SAD Accelerator Array

Output and

Monitoring

(MV, SAD, etc.)

AGU

On-Chip Memory

Ref.

Frame

Search

Window

Current

Frame

CTU

Power-Gating Control

Approximate Variant Selection Unit

User

Constraints

CPUMain Memory

System Bus

CPU

executes

the ME

algorithm

andHEVC

Memory stores

the video frames

10

Robustness for Machine Learning: News Feed

Hackers trick a Tesla into veering into the wrong lane

https://www.technologyreview.com/f/613254/hackers-trick-teslas-autopilot-into-

veering-towards-oncoming-traffic/

https://www.youtube.com/watch?v=a7L51u23YoM

Self-driving car crash in Arizona: Waymo van involved in Chandler

collision

https://www.technologyreview.com/f/613254/hackers-trick-teslas-autopilot-into-veering-towards-oncoming-traffic/https://www.youtube.com/watch?v=a7L51u23YoM

11

Risks that Arise from Adversarial Examples

FACE RECOGNITION“Accessorize to a crime: Real and stealthy attacks on state-of-the-art face recognition” [Sharif et. al. CCS 2016]

SPEECH RECOGNITION“Hidden Voice Commands”, [Carlini et. al. USENIX 2016]

MALWARE DETECTION“Automatically Evading Classifiers: A Case Study on PDF Malware Classifiers” [Xu et. al. NDSS2016 ]

TEXT CLASSIFICATION (Spam)“Crafting Adversarial Input Sequences for Recurrent Neural Networks” [Papernot et. al. 2016]

Junaid, Shafique et al. VTC 2019 Tutoria on Adv. ML

12

Reliability and Security for Machine Learning Systems

DrainSource

p+ p+

n – substrate

GateOxide Layer

Vg= – Vdd

Si HTRAP

OH+

NBTI

Aging

HCID

Process Variations Soft Errors

n+ n+

P-Well

P-Substrate

IsolationGate

+-+-

+-+- +-

+- +- +-

+-

+- Depletion Region

High-Energy Particle (Neutron or Proton)

Side Channel Attacks

1 0 1 1 0

ProcessingComputations

Mem

oryPower Supply

Machine Learning-based System Src: google ImageHardware Trojans

Neural Trojans

• M. A. Hanif, F. Khalid, R. V. W. Putra, S. Rehman, M. Shafique, “Robust Machine Learning Systems: Reliability and Security for Deep Neural Networks”, in IOLTS-2018, Platja d'Aro, Spain, pp. 257 - 260.

• F. Kriebel, S. Rehman, M. A. Hanif, F. Khalid, M. Shafique, “Robustness for Smart Cyber-Physical Systems and Internet-of-Things: From Adaptive Robustness Methods to Reliability and Security for Machine Learning”, ISVLSI-2018, Hong Kong, China, pp. 581-586.

Adversarial Attacks

Stop Sign Speed Limit (60km/h)

+

Backdoor Attacks

Src: Reiter@CCS2016

Model Stealing Attacks

Src: Tramer@UsenixSec16

P-type MOSFET

DrainSourcep+p+n – substrateGateOxide LayerVg= – VddSiHTRAPOH+

Drain

Graph3

4.4974

4.50062

4.51561

4.49764

4.51268

4.50609

4.51268

4.51268

4.49481

4.51454

4.52499

4.50218

4.53026

4.51756

4.50653

4.50218

4.48524

4.51756

4.49481

4.491

4.493

4.49188

4.4974

4.50062

4.51561

4.49764

4.5074

4.50062

4.51561

4.50764

4.51668

4.50609

4.50268

4.52268

4.50481

4.51454

4.52499

4.51218

4.53026

4.51756

4.50653

4.50218

4.48524

4.51756

4.49481

4.491

4.493

4.49188

4.51268

Sheet1

160192224256

14.49744.497400

24.500624.5006200

34.515614.5156100

44.497644.4976400

54.512684.5126851

64.506094.50609279

74.512684.512687152

84.512684.51268171111

94.494814.49481271215

104.514544.51454361324

114.524994.52499507471

124.502184.50218594580

134.530264.53026656627

144.517564.51756753676

154.506534.50653788775

164.502184.50218850821

174.485244.48524878841

184.517564.51756921902

194.494814.49481931918

204.4914.491957924

214.4934.493977950

224.491884.49188973950

234.49744.4974973967

244.500624.50062976972

254.515614.51561988975

264.497644.49764988975

274.512684.5074989980

284.506094.50062997985

294.512684.51561993983

304.512684.50764989985

314.494814.51668994995

324.514544.50609991989

334.524994.50268997991

344.502184.52268993992

354.530264.50481999997

364.517564.51454995994

374.506534.52499994996

384.502184.51218995991

394.485244.530261000997

404.517564.51756995992

414.494814.50653998990

424.4914.50218997991

434.4934.48524997996

444.491884.51756999990

454.49744.49481997997

464.500624.491999997

474.515614.493999997

484.497644.49188999997

494.512684.51268999995

504.50609999997

514.51268997998

524.512681000996

534.49481997994

544.514541000992

554.52499994998

564.5021810001000

574.53026998988

584.51756997996

594.50653999996

604.50218996999

614.48524999996

624.517561000994

634.49481997998

644.491999999

654.493998994

664.49188992994

674.4974998998

684.50062996999

694.515619991000

704.49764998996

714.51268996997

724.50609994994

734.51268999995

744.5126810001000

754.49481999994

764.51454999997

774.52499998998

784.50218985998

794.53026997982

804.517569971000

814.506539991000

824.50218998997

834.4852410001000

844.517569991000

854.494811000998

864.4919991000

874.4931000990

884.49188999996

894.4974999994

904.500621000998

914.515611000999

924.49764997999

934.512681000995

944.506099981000

954.51268998997

964.512681000999

974.49481999997

984.51454998996

994.52499997997

1004.502189961000

4.53026

4.51756

4.50653

4.50218

4.48524

4.51756

4.49481

4.491

4.493

4.49188

Sheet1

160

4.4974

4.50062

4.51561

4.49764

4.51268

4.50609

4.51268

4.51268

4.49481

4.51454

4.52499

4.50218

4.53026

4.51756

4.50653

4.50218

4.48524

4.51756

4.49481

4.491

4.493

4.49188

4.4974

4.50062

4.51561

4.49764

4.51268

4.50609

4.51268

4.51268

4.49481

4.51454

4.52499

4.50218

4.53026

4.51756

4.50653

4.50218

4.48524

4.51756

4.49481

4.491

4.493

4.49188

4.4974

4.50062

4.51561

4.49764

4.51268

Sheet2

192

4.4974

4.50062

4.51561

4.49764

4.51268

4.50609

4.51268

4.51268

4.49481

4.51454

4.52499

4.50218

4.53026

4.51756

4.50653

4.50218

4.48524

4.51756

4.49481

4.491

4.493

4.49188

4.4974

4.50062

4.51561

4.49764

4.5074

4.50062

4.51561

4.50764

4.51668

4.50609

4.50268

4.52268

4.50481

4.51454

4.52499

4.51218

4.53026

4.51756

4.50653

4.50218

4.48524

4.51756

4.49481

4.491

4.493

4.49188

4.51268

Sheet3

160192224256160192

14.49744.4974004.49744.4974

24.500624.50062004.500624.50062

34.515614.51561004.515614.51561

44.497644.49764004.497644.49764

54.512684.51268514.512684.51268

64.506094.506092794.506094.50609

74.512684.5126871524.512684.51268

84.512684.51268171111double+add4.512684.51268

94.494814.494812712154.494814.49481

104.514544.514543613244.514544.51454

114.524994.524995074714.524994.52499

124.502184.502185945804.502184.50218

134.530264.530266566274.530264.53026

144.517564.517567536764.517564.51756

154.506534.506537887754.506534.50653

164.502184.502188508214.502184.50218

174.485244.485248788414.485244.48524

184.517564.517569219024.517564.51756

194.494814.494819319184.494814.49481

204.4914.4919579244.4914.491

214.4934.4939779504.4934.493

224.491884.491889739504.49744.49188

234.49744.4974973967double+double4.500624.4974

244.500624.500629769724.515614.50062

254.515614.515619889754.497644.51561

264.497644.497649889754.512684.49764

274.512684.50749899804.506094.5074

284.506094.500629979854.512684.50062

294.512684.515619939834.512684.51561

304.512684.507649899854.494814.50764

314.494814.516689949954.514544.51668

324.514544.506099919894.524994.50609

334.524994.502689979914.502184.50268

344.502184.522689939924.530264.52268

354.530264.504819999974.517564.50481

364.517564.514549959944.506534.51454

374.506534.524999949964.502184.52499

384.502184.512189959914.485244.51218

394.485244.5302610009974.517564.53026

404.517564.517569959924.494814.51756

414.494814.506539989904.4914.50653

424.4914.502189979914.4934.50218

434.4934.48524997996tau+add4.48524

444.491884.517569999904.51756

454.49744.494819979974.49481

464.500624.4919999974.491

474.515614.4939999974.493

484.497644.491889999974.49188

494.512684.512689999954.51268

504.50609999997

514.51268997998

524.512681000996

534.49481997994

544.514541000992

554.52499994998

564.5021810001000

574.53026998988

584.51756997996

594.50653999996

604.50218996999

614.48524999996

624.517561000994tau+tau

634.49481997998

644.491999999

654.493998994

664.49188992994

674.4974998998

684.50062996999

694.5156199910004.50218

704.497649989964.48524

714.512689969974.51756

724.506099949944.49481

734.512689999954.50218

744.51268100010004.48524

754.494819999944.51756

764.514549999974.49481

774.52499998998

784.50218985998

794.53026997982

804.517569971000

814.506539991000

824.502189989974.50218

834.48524100010004.48524

844.5175699910004.51756

854.4948110009984.49481

864.49199910004.493

874.49310009904.49188

884.491889999964.4974

894.49749999944.50062

904.5006210009984.51561

914.5156110009994.49764

924.497649979994.5074

934.5126810009954.50062

944.5060999810004.51561

954.512689989974.50764

964.5126810009994.51668

974.494819999974.50609

984.514549989964.50268

994.524999979974.52268

1004.5021899610004.50481

4.530264.51454

4.517564.52499

4.506534.51218

4.502184.53026

4.485244.51756

4.517564.50653

4.494814.50218

4.4914.48524

4.4934.51756

4.491884.49481

4.491

4.493

4.49188

4.51268

Sheet3

Graph4

4.4974

4.50062

4.51561

4.49764

4.51268

4.50609

4.51268

4.51268

4.49481

4.51454

4.52499

4.50218

4.53026

4.51756

4.50653

4.50218

4.48524

4.51756

4.49481

4.491

4.493

4.4974

4.50062

4.51561

4.49764

4.51268

4.50609

4.51268

4.51268

4.49481

4.51454

4.52499

4.50218

4.53026

4.51756

4.50653

4.50218

4.48524

4.51756

4.49481

4.491

4.493

Sheet1

160192224256

14.49744.497400

24.500624.5006200

34.515614.5156100

44.497644.4976400

54.512684.5126851

64.506094.50609279

74.512684.512687152

84.512684.51268171111

94.494814.49481271215

104.514544.51454361324

114.524994.52499507471

124.502184.50218594580

134.530264.53026656627

144.517564.51756753676

154.506534.50653788775

164.502184.50218850821

174.485244.48524878841

184.517564.51756921902

194.494814.49481931918

204.4914.491957924

214.4934.493977950

224.491884.49188973950

234.49744.4974973967

244.500624.50062976972

254.515614.51561988975

264.497644.49764988975

274.512684.5074989980

284.506094.50062997985

294.512684.51561993983

304.512684.50764989985

314.494814.51668994995

324.514544.50609991989

334.524994.50268997991

344.502184.52268993992

354.530264.50481999997

364.517564.51454995994

374.506534.52499994996

384.502184.51218995991

394.485244.530261000997

404.517564.51756995992

414.494814.50653998990

424.4914.50218997991

434.4934.48524997996

444.491884.51756999990

454.49744.49481997997

464.500624.491999997

474.515614.493999997

484.497644.49188999997

494.512684.51268999995

504.50609999997

514.51268997998

524.512681000996

534.49481997994

544.514541000992

554.52499994998

564.5021810001000

574.53026998988

584.51756997996

594.50653999996

604.50218996999

614.48524999996

624.517561000994

634.49481997998

644.491999999

654.493998994

664.49188992994

674.4974998998

684.50062996999

694.515619991000

704.49764998996

714.51268996997

724.50609994994

734.51268999995

744.5126810001000

754.49481999994

764.51454999997

774.52499998998

784.50218985998

794.53026997982

804.517569971000

814.506539991000

824.50218998997

834.4852410001000

844.517569991000

854.494811000998

864.4919991000

874.4931000990

884.49188999996

894.4974999994

904.500621000998

914.515611000999

924.49764997999

934.512681000995

944.506099981000

954.51268998997

964.512681000999

974.49481999997

984.51454998996

994.52499997997

1004.502189961000

4.53026

4.51756

4.50653

4.50218

4.48524

4.51756

4.49481

4.491

4.493

4.49188

Sheet1

160

4.4974

4.50062

4.51561

4.49764

4.51268

4.50609

4.51268

4.51268

4.49481

4.51454

4.52499

4.50218

4.53026

4.51756

4.50653

4.50218

4.48524

4.51756

4.49481

4.491

4.493

4.49188

4.4974

4.50062

4.51561

4.49764

4.51268

4.50609

4.51268

4.51268

4.49481

4.51454

4.52499

4.50218

4.53026

4.51756

4.50653

4.50218

4.48524

4.51756

4.49481

4.491

4.493

4.49188

4.4974

4.50062

4.51561

4.49764

4.51268

Sheet2

192

4.4974

4.50062

4.51561

4.49764

4.51268

4.50609

4.51268

4.51268

4.49481

4.51454

4.52499

4.50218

4.53026

4.51756

4.50653

4.50218

4.48524

4.51756

4.49481

4.491

4.493

4.49188

4.4974

4.50062

4.51561

4.49764

4.5074

4.50062

4.51561

4.50764

4.51668

4.50609

4.50268

4.52268

4.50481

4.51454

4.52499

4.51218

4.53026

4.51756

4.50653

4.50218

4.48524

4.51756

4.49481

4.491

4.493

4.49188

4.51268

Sheet3

160192224256160192

14.49744.4974004.49744.4974

24.500624.50062004.500624.50062

34.515614.51561004.515614.51561

44.497644.49764004.497644.49764

54.512684.51268514.512684.51268

64.506094.506092794.506094.50609

74.512684.5126871524.512684.51268

84.512684.51268171111double+add4.512684.51268

94.494814.494812712154.494814.49481

104.514544.514543613244.514544.51454

114.524994.524995074714.524994.52499

124.502184.502185945804.502184.50218

134.530264.530266566274.530264.53026

144.517564.517567536764.517564.51756

154.506534.506537887754.506534.50653

164.502184.502188508214.502184.50218

174.485244.485248788414.485244.48524

184.517564.517569219024.517564.51756

194.494814.494819319184.494814.49481

204.4914.4919579244.4914.491

214.4934.4939779504.4934.493

224.491884.491889739504.49744.49188

234.49744.4974973967double+double4.500624.4974

244.500624.500629769724.515614.50062

254.515614.515619889754.497644.51561

264.497644.497649889754.512684.49764

274.512684.50749899804.506094.5074

284.506094.500629979854.512684.50062

294.512684.515619939834.512684.51561

304.512684.507649899854.494814.50764

314.494814.516689949954.514544.51668

324.514544.506099919894.524994.50609

334.524994.502689979914.502184.50268

344.502184.522689939924.530264.52268

354.530264.504819999974.517564.50481

364.517564.514549959944.506534.51454

374.506534.524999949964.502184.52499

384.502184.512189959914.485244.51218

394.485244.5302610009974.517564.53026

404.517564.517569959924.494814.51756

414.494814.506539989904.4914.50653

424.4914.502189979914.4934.50218

434.4934.48524997996tau+add4.48524

444.491884.517569999904.51756

454.49744.494819979974.49481

464.500624.4919999974.491

474.515614.4939999974.493

484.497644.491889999974.49188

494.512684.512689999954.51268

504.50609999997

514.51268997998

524.512681000996

534.49481997994

544.514541000992

554.52499994998

564.5021810001000

574.53026998988

584.51756997996

594.50653999996

604.50218996999

614.48524999996

624.517561000994tau+tau

634.49481997998

644.491999999

654.493998994

664.49188992994

674.4974998998

684.50062996999

694.5156199910004.50218

704.497649989964.48524

714.512689969974.51756

724.506099949944.49481

734.512689999954.50218

744.51268100010004.48524

754.494819999944.51756

764.514549999974.49481

774.52499998998

784.50218985998

794.53026997982

804.517569971000

814.506539991000

824.502189989974.50218

834.48524100010004.48524

844.5175699910004.51756

854.4948110009984.49481

864.49199910004.493

874.49310009904.49188

884.491889999964.4974

894.49749999944.50062

904.5006210009984.51561

914.5156110009994.49764

924.497649979994.5074

934.5126810009954.50062

944.5060999810004.51561

954.512689989974.50764

964.5126810009994.51668

974.494819999974.50609

984.514549989964.50268

994.524999979974.52268

1004.5021899610004.50481

4.530264.51454

4.517564.52499

4.506534.51218

4.502184.53026

4.485244.51756

4.517564.50653

4.494814.50218

4.4914.48524

4.4934.51756

4.491884.49481

4.491

4.493

4.49188

4.51268

Sheet3

Graph3

4.4974

4.50062

4.51561

4.49764

4.51268

4.50609

4.51268

4.51268

4.49481

4.51454

4.52499

4.50218

4.53026

4.51756

4.50653

4.50218

4.48524

4.51756

4.49481

4.491

4.493

4.49188

4.4974

4.50062

4.51561

4.49764

4.5074

4.50062

4.51561

4.50764

4.51668

4.50609

4.50268

4.52268

4.50481

4.51454

4.52499

4.51218

4.53026

4.51756

4.50653

4.50218

4.48524

4.51756

4.49481

4.491

4.493

4.49188

4.51268

Sheet1

160192224256

14.49744.497400

24.500624.5006200

34.515614.5156100

44.497644.4976400

54.512684.5126851

64.506094.50609279

74.512684.512687152

84.512684.51268171111

94.494814.49481271215

104.514544.51454361324

114.524994.52499507471

124.502184.50218594580

134.530264.53026656627

144.517564.51756753676

154.506534.50653788775

164.502184.50218850821

174.485244.48524878841

184.517564.51756921902

194.494814.49481931918

204.4914.491957924

214.4934.493977950

224.491884.49188973950

234.49744.4974973967

244.500624.50062976972

254.515614.51561988975

264.497644.49764988975

274.512684.5074989980

284.506094.50062997985

294.512684.51561993983

304.512684.50764989985

314.494814.51668994995

324.514544.50609991989

334.524994.50268997991

344.502184.52268993992

354.530264.50481999997

364.517564.51454995994

374.506534.52499994996

384.502184.51218995991

394.485244.530261000997

404.517564.51756995992

414.494814.50653998990

424.4914.50218997991

434.4934.48524997996

444.491884.51756999990

454.49744.49481997997

464.500624.491999997

474.515614.493999997

484.497644.49188999997

494.512684.51268999995

504.50609999997

514.51268997998

524.512681000996

534.49481997994

544.514541000992

554.52499994998

564.5021810001000

574.53026998988

584.51756997996

594.50653999996

604.50218996999

614.48524999996

624.517561000994

634.49481997998

644.491999999

654.493998994

664.49188992994

674.4974998998

684.50062996999

694.515619991000

704.49764998996

714.51268996997

724.50609994994

734.51268999995

744.5126810001000

754.49481999994

764.51454999997

774.52499998998

784.50218985998

794.53026997982

804.517569971000

814.506539991000

824.50218998997

834.4852410001000

844.517569991000

854.494811000998

864.4919991000

874.4931000990

884.49188999996

894.4974999994

904.500621000998

914.515611000999

924.49764997999

934.512681000995

944.506099981000

954.51268998997

964.512681000999

974.49481999997

984.51454998996

994.52499997997

1004.502189961000

4.53026

4.51756

4.50653

4.50218

4.48524

4.51756

4.49481

4.491

4.493

4.49188

Sheet1

160

4.4974

4.50062

4.51561

4.49764

4.51268

4.50609

4.51268

4.51268

4.49481

4.51454

4.52499

4.50218

4.53026

4.51756

4.50653

4.50218

4.48524

4.51756

4.49481

4.491

4.493

4.49188

4.4974

4.50062

4.51561

4.49764

4.51268

4.50609

4.51268

4.51268

4.49481

4.51454

4.52499

4.50218

4.53026

4.51756

4.50653

4.50218

4.48524

4.51756

4.49481

4.491

4.493

4.49188

4.4974

4.50062

4.51561

4.49764

4.51268

Sheet2

192

4.4974

4.50062

4.51561

4.49764

4.51268

4.50609

4.51268

4.51268

4.49481

4.51454

4.52499

4.50218

4.53026

4.51756

4.50653

4.50218

4.48524

4.51756

4.49481

4.491

4.493

4.49188

4.4974

4.50062

4.51561

4.49764

4.5074

4.50062

4.51561

4.50764

4.51668

4.50609

4.50268

4.52268

4.50481

4.51454

4.52499

4.51218

4.53026

4.51756

4.50653

4.50218

4.48524

4.51756

4.49481

4.491

4.493

4.49188

4.51268

Sheet3

160192224256160192

14.49744.4974004.49744.4974

24.500624.50062004.500624.50062

34.515614.51561004.515614.51561

44.497644.49764004.497644.49764

54.512684.51268514.512684.51268

64.506094.506092794.506094.50609

74.512684.5126871524.512684.51268

84.512684.51268171111double+add4.512684.51268

94.494814.494812712154.494814.49481

104.514544.514543613244.514544.51454

114.524994.524995074714.524994.52499

124.502184.502185945804.502184.50218

134.530264.530266566274.530264.53026

144.517564.517567536764.517564.51756

154.506534.506537887754.506534.50653

164.502184.502188508214.502184.50218

174.485244.485248788414.485244.48524

184.517564.517569219024.517564.51756

194.494814.494819319184.494814.49481

204.4914.4919579244.4914.491

214.4934.4939779504.4934.493

224.491884.491889739504.49744.49188

234.49744.4974973967double+double4.500624.4974

244.500624.500629769724.515614.50062

254.515614.515619889754.497644.51561

264.497644.497649889754.512684.49764

274.512684.50749899804.506094.5074

284.506094.500629979854.512684.50062

294.512684.515619939834.512684.51561

304.512684.507649899854.494814.50764

314.494814.516689949954.514544.51668

324.514544.506099919894.524994.50609

334.524994.502689979914.502184.50268

344.502184.522689939924.530264.52268

354.530264.504819999974.517564.50481

364.517564.514549959944.506534.51454

374.506534.524999949964.502184.52499

384.502184.512189959914.485244.51218

394.485244.5302610009974.517564.53026

404.517564.517569959924.494814.51756

414.494814.506539989904.4914.50653

424.4914.502189979914.4934.50218

434.4934.48524997996tau+add4.48524

444.491884.517569999904.51756

454.49744.494819979974.49481

464.500624.4919999974.491

474.515614.4939999974.493

484.497644.491889999974.49188

494.512684.512689999954.51268

504.50609999997

514.51268997998

524.512681000996

534.49481997994

544.514541000992

554.52499994998

564.5021810001000

574.53026998988

584.51756997996

594.50653999996

604.50218996999

614.48524999996

624.517561000994tau+tau

634.49481997998

644.491999999

654.493998994

664.49188992994

674.4974998998

684.50062996999

694.5156199910004.50218

704.497649989964.48524

714.512689969974.51756

724.506099949944.49481

734.512689999954.50218

744.51268100010004.48524

754.494819999944.51756

764.514549999974.49481

774.52499998998

784.50218985998

794.53026997982

804.517569971000

814.506539991000

824.502189989974.50218

834.48524100010004.48524

844.5175699910004.51756

854.4948110009984.49481

864.49199910004.493

874.49310009904.49188

884.491889999964.4974

894.49749999944.50062

904.5006210009984.51561

914.5156110009994.49764

924.497649979994.5074

934.5126810009954.50062

944.5060999810004.51561

954.512689989974.50764

964.5126810009994.51668

974.494819999974.50609

984.514549989964.50268

994.524999979974.52268

1004.5021899610004.50481

4.530264.51454

4.517564.52499

4.506534.51218

4.502184.53026

4.485244.51756

4.517564.50653

4.494814.50218

4.4914.48524

4.4934.51756

4.491884.49481

4.491

4.493

4.49188

4.51268

Sheet3

Graph3

4.4974

4.50062

4.51561

4.49764

4.51268

4.50609

4.51268

4.51268

4.49481

4.51454

4.52499

4.50218

4.53026

4.51756

4.50653

4.50218

4.48524

4.51756

4.49481

4.491

4.493

4.49188

4.4974

4.50062

4.51561

4.49764

4.5074

4.50062

4.51561

4.50764

4.51668

4.50609

4.50268

4.52268

4.50481

4.51454

4.52499

4.51218

4.53026

4.51756

4.50653

4.50218

4.48524

4.51756

4.49481

4.491

4.493

4.49188

4.51268

Sheet1

160192224256

14.49744.497400

24.500624.5006200

34.515614.5156100

44.497644.4976400

54.512684.5126851

64.506094.50609279

74.512684.512687152

84.512684.51268171111

94.494814.49481271215

104.514544.51454361324

114.524994.52499507471

124.502184.50218594580

134.530264.53026656627

144.517564.51756753676

154.506534.50653788775

164.502184.50218850821

174.485244.48524878841

184.517564.51756921902

194.494814.49481931918

204.4914.491957924

214.4934.493977950

224.491884.49188973950

234.49744.4974973967

244.500624.50062976972

254.515614.51561988975

264.497644.49764988975

274.512684.5074989980

284.506094.50062997985

294.512684.51561993983

304.512684.50764989985

314.494814.51668994995

324.514544.50609991989

334.524994.50268997991

344.502184.52268993992

354.530264.50481999997

364.517564.51454995994

374.506534.52499994996

384.502184.51218995991

394.485244.530261000997

404.517564.51756995992

414.494814.50653998990

424.4914.50218997991

434.4934.48524997996

444.491884.51756999990

454.49744.49481997997

464.500624.491999997

474.515614.493999997

484.497644.49188999997

494.512684.51268999995

504.50609999997

514.51268997998

524.512681000996

534.49481997994

544.514541000992

554.52499994998

564.5021810001000

574.53026998988

584.51756997996

594.50653999996

604.50218996999

614.48524999996

624.517561000994

634.49481997998

644.491999999

654.493998994

664.49188992994

674.4974998998

684.50062996999

694.515619991000

704.49764998996

714.51268996997

724.50609994994

734.51268999995

744.5126810001000

754.49481999994

764.51454999997

774.52499998998

784.50218985998

794.53026997982

804.517569971000

814.506539991000

824.50218998997

834.4852410001000

844.517569991000

854.494811000998

864.4919991000

874.4931000990

884.49188999996

894.4974999994

904.500621000998

914.515611000999

924.49764997999

934.512681000995

944.506099981000

954.51268998997

964.512681000999

974.49481999997

984.51454998996

994.52499997997

1004.502189961000

4.53026

4.51756

4.50653

4.50218

4.48524

4.51756

4.49481

4.491

4.493

4.49188

Sheet1

160

4.4974

4.50062

4.51561

4.49764

4.51268

4.50609

4.51268

4.51268

4.49481

4.51454

4.52499

4.50218

4.53026

4.51756

4.50653

4.50218

4.48524

4.51756

4.49481

4.491

4.493

4.49188

4.4974

4.50062

4.51561

4.49764

4.51268

4.50609

4.51268

4.51268

4.49481

4.51454

4.52499

4.50218

4.53026

4.51756

4.50653

4.50218

4.48524

4.51756

4.49481

4.491

4.493

4.49188

4.4974

4.50062

4.51561

4.49764

4.51268

Sheet2

192

4.4974

4.50062

4.51561

4.49764

4.51268

4.50609

4.51268

4.51268

4.49481

4.51454

4.52499

4.50218

4.53026

4.51756

4.50653

4.50218

4.48524

4.51756

4.49481

4.491

4.493

4.49188

4.4974

4.50062

4.51561

4.49764

4.5074

4.50062

4.51561

4.50764

4.51668

4.50609

4.50268

4.52268

4.50481

4.51454

4.52499

4.51218

4.53026

4.51756

4.50653

4.50218

4.48524

4.51756

4.49481

4.491

4.493

4.49188

4.51268

Sheet3

160192224256160192

14.49744.4974004.49744.4974

24.500624.50062004.500624.50062

34.515614.51561004.515614.51561

44.497644.49764004.497644.49764

54.512684.51268514.512684.51268

64.506094.506092794.506094.50609

74.512684.5126871524.512684.51268

84.512684.51268171111double+add4.512684.51268

94.494814.494812712154.494814.49481

104.514544.514543613244.514544.51454

114.524994.524995074714.524994.52499

124.502184.502185945804.502184.50218

134.530264.530266566274.530264.53026

144.517564.517567536764.517564.51756

154.506534.506537887754.506534.50653

164.502184.502188508214.502184.50218

174.485244.485248788414.485244.48524

184.517564.517569219024.517564.51756

194.494814.494819319184.494814.49481

204.4914.4919579244.4914.491

214.4934.4939779504.4934.493

224.491884.491889739504.49744.49188

234.49744.4974973967double+double4.500624.4974

244.500624.500629769724.515614.50062

254.515614.515619889754.497644.51561

264.497644.497649889754.512684.49764

274.512684.50749899804.506094.5074

284.506094.500629979854.512684.50062

294.512684.515619939834.512684.51561

304.512684.507649899854.494814.50764

314.494814.516689949954.514544.51668

324.514544.506099919894.524994.50609

334.524994.502689979914.502184.50268

344.502184.522689939924.530264.52268

354.530264.504819999974.517564.50481

364.517564.514549959944.506534.51454

374.506534.524999949964.502184.52499

384.502184.512189959914.485244.51218

394.485244.5302610009974.517564.53026

404.517564.517569959924.494814.51756

414.494814.506539989904.4914.50653

424.4914.502189979914.4934.50218

434.4934.48524997996tau+add4.48524

444.491884.517569999904.51756

454.49744.494819979974.49481

464.500624.4919999974.491

474.515614.4939999974.493

484.497644.491889999974.49188

494.512684.512689999954.51268

504.50609999997

514.51268997998

524.512681000996

534.49481997994

544.514541000992

554.52499994998

564.5021810001000

574.53026998988

584.51756997996

594.50653999996

604.50218996999

614.48524999996

624.517561000994tau+tau

634.49481997998

644.491999999

654.493998994

664.49188992994

674.4974998998

684.50062996999

694.5156199910004.50218

704.497649989964.48524

714.512689969974.51756

724.506099949944.49481

734.512689999954.50218

744.51268100010004.48524

754.494819999944.51756

764.514549999974.49481

774.52499998998

784.50218985998

794.53026997982

804.517569971000

814.506539991000

824.502189989974.50218

834.48524100010004.48524

844.5175699910004.51756

854.4948110009984.49481

864.49199910004.493

874.49310009904.49188

884.491889999964.4974

894.49749999944.50062

904.5006210009984.51561

914.5156110009994.49764

924.497649979994.5074

934.5126810009954.50062

944.5060999810004.51561

954.512689989974.50764

964.5126810009994.51668

974.494819999974.50609

984.514549989964.50268

994.524999979974.52268

1004.5021899610004.50481

4.530264.51454

4.517564.52499

4.506534.51218

4.502184.53026

4.485244.51756

4.517564.50653

4.494814.50218

4.4914.48524

4.4934.51756

4.491884.49481

4.491

4.493

4.49188

4.51268

Sheet3

Graph4

4.4974

4.50062

4.51561

4.49764

4.51268

4.50609

4.51268

4.51268

4.49481

4.51454

4.52499

4.50218

4.53026

4.51756

4.50653

4.50218

4.48524

4.51756

4.49481

4.491

4.493

4.4974

4.50062

4.51561

4.49764

4.51268

4.50609

4.51268

4.51268

4.49481

4.51454

4.52499

4.50218

4.53026

4.51756

4.50653

4.50218

4.48524

4.51756

4.49481

4.491

4.493

Sheet1

160192224256

14.49744.497400

24.500624.5006200

34.515614.5156100

44.497644.4976400

54.512684.5126851

64.506094.50609279

74.512684.512687152

84.512684.51268171111

94.494814.49481271215

104.514544.51454361324

114.524994.52499507471

124.502184.50218594580

134.530264.53026656627

144.517564.51756753676

154.506534.50653788775

164.502184.50218850821

174.485244.48524878841

184.517564.51756921902

194.494814.49481931918

204.4914.491957924

214.4934.493977950

224.491884.49188973950

234.49744.4974973967

244.500624.50062976972

254.515614.51561988975

264.497644.49764988975

274.512684.5074989980

284.506094.50062997985

294.512684.51561993983

304.512684.50764989985

314.494814.51668994995

324.514544.50609991989

334.524994.50268997991

344.502184.52268993992

354.530264.50481999997

364.517564.51454995994

374.506534.52499994996

384.502184.51218995991

394.485244.530261000997

404.517564.51756995992

414.494814.50653998990

424.4914.50218997991

434.4934.48524997996

444.491884.51756999990

454.49744.49481997997

464.500624.491999997

474.515614.493999997

484.497644.49188999997

494.512684.51268999995

504.50609999997

514.51268997998

524.512681000996

534.49481997994

544.514541000992

554.52499994998

564.5021810001000

574.53026998988

584.51756997996

594.50653999996

604.50218996999

614.48524999996

624.517561000994

634.49481997998

644.491999999

654.493998994

664.49188992994

674.4974998998

684.50062996999

694.515619991000

704.49764998996

714.51268996997

724.50609994994

734.51268999995

744.5126810001000

754.49481999994

764.51454999997

774.52499998998

784.50218985998

794.53026997982

804.517569971000

814.506539991000

824.50218998997

834.4852410001000

844.517569991000

854.494811000998

864.4919991000

874.4931000990

884.49188999996

894.4974999994

904.500621000998

914.515611000999

924.49764997999

934.512681000995

944.506099981000

954.51268998997

964.512681000999

974.49481999997

984.51454998996

994.52499997997

1004.502189961000

4.53026

4.51756

4.50653

4.50218

4.48524

4.51756

4.49481

4.491

4.493

4.49188

Sheet1

160

4.4974

4.50062

4.51561

4.49764

4.51268

4.50609

4.51268

4.51268

4.49481

4.51454

4.52499

4.50218

4.53026

4.51756

4.50653

4.50218

4.48524

4.51756

4.49481

4.491

4.493

4.49188

4.4974

4.50062

4.51561

4.49764

4.51268

4.50609

4.51268

4.51268

4.49481

4.51454

4.52499

4.50218

4.53026

4.51756

4.50653

4.50218

4.48524

4.51756

4.49481

4.491

4.493

4.49188

4.4974

4.50062

4.51561

4.49764

4.51268

Sheet2

192

4.4974

4.50062

4.51561

4.49764

4.51268

4.50609

4.51268

4.51268

4.49481

4.51454

4.52499

4.50218

4.53026

4.51756

4.50653

4.50218

4.48524

4.51756

4.49481

4.491

4.493

4.49188

4.4974

4.50062

4.51561

4.49764

4.5074

4.50062

4.51561

4.50764

4.51668

4.50609

4.50268

4.52268

4.50481

4.51454

4.52499

4.51218

4.53026

4.51756

4.50653

4.50218

4.48524

4.51756

4.49481

4.491

4.493

4.49188

4.51268

Sheet3

160192224256160192

14.49744.4974004.49744.4974

24.500624.50062004.500624.50062

34.515614.51561004.515614.51561

44.497644.49764004.497644.49764

54.512684.51268514.512684.51268

64.506094.506092794.506094.50609

74.512684.5126871524.512684.51268

84.512684.51268171111double+add4.512684.51268

94.494814.494812712154.494814.49481

104.514544.514543613244.514544.51454

114.524994.524995074714.524994.52499

124.502184.502185945804.502184.50218

134.530264.530266566274.530264.53026

144.517564.517567536764.517564.51756

154.506534.506537887754.506534.50653

164.502184.502188508214.502184.50218

174.485244.485248788414.485244.48524

184.517564.517569219024.517564.51756

194.494814.494819319184.494814.49481

204.4914.4919579244.4914.491

214.4934.4939779504.4934.493

224.491884.491889739504.49744.49188

234.49744.4974973967double+double4.500624.4974

244.500624.500629769724.515614.50062

254.515614.515619889754.497644.51561

264.497644.497649889754.512684.49764

274.512684.50749899804.506094.5074

284.506094.500629979854.512684.50062

294.512684.515619939834.512684.51561

304.512684.507649899854.494814.50764

314.494814.516689949954.514544.51668

324.514544.506099919894.524994.50609

334.524994.502689979914.502184.50268

344.502184.522689939924.530264.52268

354.530264.504819999974.517564.50481

364.517564.514549959944.506534.51454

374.506534.524999949964.502184.52499

384.502184.512189959914.485244.51218

394.485244.5302610009974.517564.53026

404.517564.517569959924.494814.51756

414.494814.506539989904.4914.50653

424.4914.502189979914.4934.50218

434.4934.48524997996tau+add4.48524

444.491884.517569999904.51756

454.49744.494819979974.49481

464.500624.4919999974.491

474.515614.4939999974.493

484.497644.491889999974.49188

494.512684.512689999954.51268

504.50609999997

514.51268997998

524.512681000996

534.49481997994

544.514541000992

554.52499994998

564.5021810001000

574.53026998988

584.51756997996

594.50653999996

604.50218996999

614.48524999996

624.517561000994tau+tau

634.49481997998

644.491999999

654.493998994

664.49188992994

674.4974998998

684.50062996999

694.5156199910004.50218

704.497649989964.48524

714.512689969974.51756

724.506099949944.49481

734.512689999954.50218

744.51268100010004.48524

754.494819999944.51756

764.514549999974.49481

774.52499998998

784.50218985998

794.53026997982

804.517569971000

814.506539991000

824.502189989974.50218

834.48524100010004.48524

844.5175699910004.51756

854.4948110009984.49481

864.49199910004.493

874.49310009904.49188

884.491889999964.4974

894.49749999944.50062

904.5006210009984.51561

914.5156110009994.49764

924.497649979994.5074

934.5126810009954.50062

944.5060999810004.51561

954.512689989974.50764

964.5126810009994.51668

974.494819999974.50609

984.514549989964.50268

994.524999979974.52268

1004.5021899610004.50481

4.530264.51454

4.517564.52499

4.506534.51218

4.502184.53026

4.485244.51756

4.517564.50653

4.494814.50218

4.4914.48524

4.4934.51756

4.491884.49481

4.491

4.493

4.49188

4.51268

Sheet3

13

Details: Refer to Our Works

Tutorial on "Adversarial ML and Vehicular Networks: Strategies for Attack and Defense" at IEEE 89th Vehicular Technology Conference (VTC).

Selectd Research Papers DAC’19: "Building Robust Machine Learning Systems: Current

Progress, Research Challenges, and Opportunities". ICML’19 Workshop on Robust ML: “CapsAttacks: Robust and

Imperceptible Adversarial Attacks on Capsule Networks” DATE’19: “FAdeML: Understanding the Impact of Pre-

Processing Noise Filtering on Adversarial Machine Learning” IOLTS’19: “TrISec: Training Data-Unaware Imperceptible

Security Attacks on Deep Neural Networks”. IOLTS’19: “QuSecNets: Quantization-based Defense

Mechanism for Securing Deep Neural Network against Adversarial Attacks”.

14

Details: Refer to Our Works

Tutorial on "Adversarial ML and Vehicular Networks: Strategies for Attack and Defense" at IEEE 89th Vehicular Technology Conference (VTC).

Selectd Research Papers DAC’19: "Building Robust Machine Learning Systems: Current

Progress, Research Challenges, and Opportunities". ICML’19 Workshop on Robust ML: “CapsAttacks: Robust and

Imperceptible Adversarial Attacks on Capsule Networks” DATE’19: “FAdeML: Understanding the Impact of Pre-

Processing Noise Filtering on Adversarial Machine Learning” IOLTS’19: “TrISec: Training Data-Unaware Imperceptible

Security Attacks on Deep Neural Networks”. IOLTS’19: “QuSecNets: Quantization-based Defense

Mechanism for Securing Deep Neural Network against Adversarial Attacks”.

RTML Workshop at ITC 2019“Open for Submissions”

15

Data Poisoning Attacks during Training

T2’

T1

T2

T1 +T2’

T = T1 + T2

Trained Model

Training

InferenceStop

Attack 1: In this attack certain portion of the original dataset is intrudedwhich is then used for training.

60km/h

T = T1 + T2

T2

T1

T2’

T1 + T2 + T2’

Trained Model

Training

InferenceStop 60km/h

Attack 2: Instead of perturbing the original dataset, in this attack we extend thedataset with intruded samples which is then used for training.

Faiq, Shafique et al. @ FIT 2018, DSD’18, IOLTS 2019

16

Data Poisoning Attacks: Training16N0: No Noise N1: 1% Noise N2: .% Noise N1: 5% Noise

020406080

100

N0 N1 N3 N5

Top

Accu

racy

N0 N1 N3 N5

Stop

Yield

Dangerous Curve

Priority Road

Speed Limit 60

020406080

100

N0 N1 N3 N5

Top

Accu

racy

N0 N1 N3 N5

Turn Right

Dangerous Curve

Yield

Stop

Tun Left

Attack 1 Attack 2

Attack 1 Attack 2


17

Data Poisoning Attacks: Training17N0: No Noise N1: 1% Noise N2: .% Noise N1: 5% Noise

020406080

100

N0 N1 N3 N5

Top

Accu

racy

N0 N1 N3 N5

Stop

Yield

Dangerous Curve

Priority Road

Speed Limit 60

020406080

100

N0 N1 N3 N5

Top

Accu

racy

N0 N1 N3 N5

Turn Right

Dangerous Curve

Yield

Stop

Tun Left

Attack 1 Attack 2

Attack 1 Attack 2


To develop a successful (less impact on validation accuracy)Data Poisoning Attack, append the intruded samples in original

dataset instead of changing the original dataset

Limitations:1. Require the access to the complete training dataset2. The attack Noise is perceptible.

18

Adversarial Attacks on ML-Systems

Specially crafted imperceptible noise to misclassify the ML System Imperceptibility Robustness

Output Label = Speed Limit

(60km/h)

Buffer

Preprocessing Noise filters

DNN

Input Label = Stop

Perturbed Image

Environmental Noise and Variations

Challenges• Ensure the imperceptibility• Handling the impact of environmental Noise

Subjective Analysis

Faiq, Shafique et al. @ IOLTS 2019

19

Adversarial Attacks

DNN

Training Dataset

DNN

Trained Model

Network Parameters

W11W21

Wn1

W12W22

Wn2

W1nW2n

Wnn

b1b2

bn

Gradient-basedAdversarial Attacks

(a) White-Box Attacks

Adversarial Example

DN

N Architecture

Training

(b) Black-Box Attacks

DNN

Training Dataset

DNN

Trained Model

Probability vectorScore-based

Decision-based Output Label

Model StealingDNN

Network Parameters

W11

W21

Wn1

W12

W22

Wn2

W1n

W2n

Wnn

b1

b2

bn

Gradient-basedAdversarial Attacks

Adversarial Example

Adversarial Example

Transfer Attacks

DNN Architecture

Training


20

TrISec: Training Data Unaware Adversarial Attack

Deep Neural Network

Perturbed Image(x+δ)

Cost Functi

on

𝐶𝐶 = �𝑗𝑗=0

𝑛𝑛𝐿𝐿−1

(𝑎𝑎𝑗𝑗𝐿𝐿 − 𝑦𝑦𝑗𝑗)2

Current Distribution

Targeted Distribution

Compute Correlation

(CR)

PASS

𝜕𝜕𝐶𝐶

𝜕𝜕𝑎𝑎𝑗𝑗(𝐿𝐿) = 2 �

𝑗𝑗=0


(𝑎𝑎𝑗𝑗𝐿𝐿 − 𝑦𝑦𝑗𝑗)

𝜕𝜕𝐶𝐶

𝜕𝜕𝑎𝑎𝑗𝑗(𝐿𝐿−1) = �

𝑗𝑗=0


𝑤𝑤𝑗𝑗𝑗𝑗𝐿𝐿 𝜎𝜎𝜎 𝑧𝑧𝑗𝑗

𝐿𝐿−1 𝜕𝜕𝐶𝐶

𝜕𝜕𝑎𝑎𝑗𝑗(𝐿𝐿)

Compute the effect of cost (C) on activation function (a)

FAIL

Compute structural similarity index (SS)

Original Image

(x)

Attack Image

PASS FAIL

PASS

0

50

100

A B C D E

0

50

100

A B C D E

Targeted Class or Targeted Classification

Probability

FAIL

Update Noise /

Perturbation (𝛿𝛿)

𝑎𝑎(𝐿𝐿)

𝑦𝑦

0.95 ≤ 𝐶𝐶𝐶𝐶 ≤ 1 𝛿𝛿 = 𝛿𝛿 × (1 − 𝐶𝐶𝐶𝐶)

𝛿𝛿 = 𝛿𝛿 × (1 − 𝑆𝑆𝑆𝑆)

C ≤ ε

0.99 ≤ 𝑆𝑆𝑆𝑆 ≤ 1

F. Khalid, M. A. Hanif, S. Rehman, M. Shafique, “TrISec: Training Data-Unaware Imperceptible Security Attacks on Machine Learning Modules of Autonomous Vehicles”, IOLTS2019. [Link: https://arxiv.org/abs/1811.01031]

+

Perturbation (𝛿𝛿)

G2: Imperceptibility

G1: Misclassification

https://arxiv.org/abs/1811.01031

21

Imperceptible Data Poisoning Attacks during Inference

Key Insights:• Instead of training data samples, the probability vector at the

output can be used to optimize the attack noise.• To ensure the imperceptibility, additional similarity and correlation

checks are required

I0: No Noise I1: 1st Iteration I25: 25Kth Iteration I50: 50Kth Iteration

0.80.850.90.951

0255075

100

I0 I1 I25 I50

SSI,

CR

Top

5 Ac

cura

cy

StopYieldDangerous CurvePriority RoadSpeed Limit 60CRSSI

Top 5 Accuracy, Correlation

Coefficient (CR) and Structural Similarity

Index (SSI)

F. Khalid, M. A. Hanif, S. Rehman, M. Shafique, “TrISec: Training Data-Unaware Imperceptible Security Attacks on Machine Learning Modules of Autonomous Vehicles”, IOLTS 2019. [Link: https://arxiv.org/abs/1811.01031 ]

https://arxiv.org/abs/1811.01031

22

Practical Implications of Adversarial Perturbations22

Input Label = Stop Output Label =

Speed Limit (60km/h)


Perturbed Image

Attacker can physically insert the perturbations in sensor’s data

DNN 020406080

100

A B C D E

Buffe

r

Subjective Analysis




Perturbed Image

Attacker has access to the output of the pre-processing module

DNN 020406080

100

A B C D E

Buffe

r

Subjective Analysis

Faiq, Shafique et al. @ DATE 2019




Perturbed Image

Attacker has access to the input of the pre-processing module

DNN 020406080

100

A B C D E

Buffe

r

Subjective Analysis

23

Potential Defenses: FAdeML

Cost Function 𝑓𝑓(𝑐𝑐𝑐𝑐𝑐𝑐𝑐𝑐)

Buffer

Preprocessing Noise Filters

0

30

60

90

A B C D E

020406080

100

A B C D E

Adversarial Attack LibraryI-FGSM, JSMA, FGSM, TrISec

Optimization AlgorithmSpecific to selected

Adversarial Attack

Cost Function 𝑓𝑓(𝑐𝑐𝑐𝑐𝑐𝑐𝑐𝑐)

Input samples

Adversarial Examples

Reference Sample(s)

020406080

100

A B C D E

020406080

100

A B C D E

75.68%

Stop Stop Sign

Stop Stop Sign

Stop 60km/h

Stop 60km/h

FGSM I-FGSM JSMA

85.61% 89.24% 92.58%

TrISec

78.45% 72.14% 62.15% 68.68%

68.45% 72.36% 74.28% 75.12%

99.47% 99.47% 99.47% 99.47%DNN

DNN

DNN

DNN

Analysis 1: Attacker has direct access to the input of DNN, i.e., access to the output of

the preprocessing module.

Analysis 2: Attacker does not direct access to the input of DNN and does not have knowledge of preprocessing module.

Analysis 3: Attacker does not direct access to the input of DNN and have the complete

knowledge of the preprocessing module.Faiq, Shafique et al. @ DATE 2019

24

Defenses Against Adversarial ExamplesGoal Increase the perceptibility of the attack noise

Decrease the robustness of the attack, i.e., nullify the effects of imperceptible attack Noise

Deer

Airplane Automobile

Deer

Before Defense After Defense

Before Defense After Defense

Automobile

Automobile

Original

Original


25

IOLTS’19: Effects of Quantization on Adversarial ML25

A

A

A

Correct Classification of Clean ImagesIncorrect Classification of Perturbed Images

Correct Classification of Perturbed Images

Clean Images

Perturbed Image

Quantized Clean ImagesQuantized Perturbed ImageNormal Process

Adversarial Attacks

Claverhans(FGSM, CW,

JSMA) CNNQuantization Layer (QL)

CNN

CIFAR-10

Perturbed Image Perturbed Quantized

Images

P(Ship) = 0.9037

Clean Image

P(Airplane) = 0.7969



Clean Quantized

Images

P(3) = 0.862

P(8) = 0.457

P(1) = 1.0

P(3) = 0.911

P(8) = 0.524

P(3) = 0.897

P(Dog) = 0.75

P(Automobile) = 0.91

P(Deer) = 0.97

P(Dog) = 0.69

P(Dog) = 0.34

P(3) = 0.55

WithQuantization Layer

Without Quantization Layer



No Attack

FGSM Attack

C&W Attack

26

IOLTS’19: Effects of Quantization on Adversarial ML26

A

A

A

Correct Classification of Clean ImagesIncorrect Classification of Perturbed Images

Correct Classification of Perturbed Images

Clean Images

Perturbed Image

Quantized Clean ImagesQuantized Perturbed ImageNormal Process

Adversarial Attacks

Claverhans(FGSM, CW,

JSMA) CNNQuantization Layer (QL)

CNN

CIFAR-10

Perturbed Image Perturbed Quantized

Images

P(Ship) = 0.9037

Clean Image




Clean Quantized

Images

P(3) = 0.862

P(8) = 0.457

P(1) = 1.0

P(3) = 0.911

P(8) = 0.524

P(3) = 0.897

P(Dog) = 0.75

P(Automobile) = 0.91

P(Deer) = 0.97

P(Dog) = 0.69

P(Dog) = 0.34

P(3) = 0.55





No Attack

FGSM Attack

C&W Attack

Challenging Question

Can quantization as an input transformation be used to increase the perceptibility of the adversarial noise

while preserving the classification accuracy?

27

IOLTS’19: QuSecNets – Quantization-based Defense 27

Constant

Trainable

Quantization

QL

TrainingOnly for CQ Only for TQ

For Both CQ and TQ

QuSecNets

Training Dataset

DNN Model

QL: Quantization Layer

n = Number of Quantization Levels𝑐𝑐𝑗𝑗 is trainable for Trainable quantization

+

𝑆𝑆(𝑥𝑥, 𝑐𝑐1)

𝑆𝑆(𝑥𝑥, 𝑐𝑐2)

𝑆𝑆(𝑥𝑥, 𝑐𝑐𝑛𝑛−1)

𝑦𝑦1

𝑦𝑦2

𝑦𝑦𝑛𝑛−1

𝑥𝑥

1𝑛𝑛 − 1

𝑦𝑦

Error Injection

Error Resilience Analysis

Selecting “n”

Maximum Imperceptible Perturbations

Maximum Tolerable Noise


28

Experimental Results28

92

94

96

98

100

0 0.1 0.2 0.3

Q_2

Q_3

Q_4

Q_5

Q_6Maximum allowed perturbations in a single pixel (epsilon)

Clas

sific

atio

n A

ccur

acy

unde

r Att

ack

Scen

ario Trainable

QuantizationConstant Quantization

Maximum allowed perturbations in a single pixel (epsilon)

Clas

sific

atio

n Ac

cura

cy

unde

r Att

ack

Scen

ario

0

50

100

0 0.1 0.2 0.3

No Defense

Bird Deer

FrogShip

Airplane

Airplane

Automobile

Trainable Quantization

Constant Quantization

No Defense

Trainable Quantization performs better than Constant Quantization

Increase the perceptibility of the adversarial noise

Increase the classification accuracy under attack

FSGM Attack

JSMA Attack



Classification Accuracy under Attack Scenario

1

Q_200.10.20.398.898.6798.598.5Q_300.10.20.399.198.8398.2597.67Q_400.10.20.399.398.6397.4996Q_500.10.20.399.398.596.6894.61Q_600.10.20.399.398.2594.7192.69

Trainable Quantization

Constant Quantization


Classification Accuracy under Attack Scenario

No Defense

00.10.20.310010.521.25000.10.20.399.297.1194.792.1800.10.20.398.9798.4697.7797.1

29

The imperceptibility of the adversarial noise increases with the increase in number of the allowed queries.

Input Label = Stop

Preprocessing

DNNP(1)P(2)P(3)

P(n)

Probability Vector

Black-boxP(1)

Top-1 Probability

P(1)

Top-1 Label

Decision–based Attacks

Clean ImagesCIFAR-10 Dataset

Adversarial examples when number of queries (Q) is

restricted to 1000


restricted to 10000

Case I Case II Case III Case IV Case V

0

50

100

150

200

250

300


Pertu

rbat

ion

Nor

m

Q = 1000 Q = 10000

94.9

275.

8

293.

1

269.

2

1.51

2.44

1.54

3.47

0.7913

Decision-based Adversarial Attacks

30


Input Label = Stop

Preprocessing

DNNP(1)P(2)P(3)

P(n)

Probability Vector

Black-boxP(1)

Top-1 Probability

P(1)

Top-1 Label




restricted to 1000


restricted to 10000


0

50

100

150

200

250

300


Pertu

rbat

ion

Nor

m

Q = 1000 Q = 10000

94.9

275.

8

293.

1

269.

2

1.51

2.44

1.54

3.47

0.7913


Can new decision-based attacks be devised that can operate successfully in real-time resource-

constrained applications while ensuring imperceptibility and robustness of attacks?


31


Input Label = Stop

Preprocessing

DNNP(1)P(2)P(3)

P(n)

Probability Vector

Black-boxP(1)

Top-1 Probability

P(1)

Top-1 Label




restricted to 1000


restricted to 10000


0

50

100

150

200

250

300


Pertu

rbat

ion

Nor

m

Q = 1000 Q = 10000

94.9

275.

8

293.

1

269.

2

1.51

2.44

1.54

3.47

0.7913


Can new decision-based attacks be devised that can operate successfully in real-time resource-

constrained applications while ensuring imperceptibility and robustness of attacks?


F. Khalid, M. Shafique et al., “RED-Attack: Resource Efficient Decision based Attack

for Machine Learning”, [Link: https://arxiv.org/abs/1901.10258]

32

My Research Team and CollaboratorsPost-Docs and PhDs

Collaborators

32

MS/BS Students

Previous Students

Computer Architecture and RobustEnergy-Efficient Technologies Group

CAREech

Thank you!Questions?

M. [email protected]

Security for Machine Learning Systems �Attacks, Defenses, & ReconfigurabilityWho Ruled the World!Smart Cyber Physical Systems & Internet-of-ThingsSmart CPS & IoT => The Big Data Processing Challenge!Smart CPS & IoT => The Big Data Processing Challenge!ML Applications => requiring High Efficiency GainsMachine Learning and the Computing Efficiency Gap!Machine Learning and the Computing Efficiency Gap!Brain-Inspired Computing: [email protected] for Machine Learning: News FeedRisks that Arise from Adversarial ExamplesReliability and Security for Machine Learning SystemsDetails: Refer to Our WorksDetails: Refer to Our WorksData Poisoning Attacks during TrainingData Poisoning Attacks: TrainingData Poisoning Attacks: TrainingAdversarial Attacks on ML-SystemsAdversarial AttacksTrISec: Training Data Unaware Adversarial AttackImperceptible Data Poisoning Attacks during InferencePractical Implications of Adversarial PerturbationsPotential Defenses: FAdeMLDefenses Against Adversarial ExamplesIOLTS’19: Effects of Quantization on Adversarial MLIOLTS’19: Effects of Quantization on Adversarial MLIOLTS’19: QuSecNets – Quantization-based Defense Experimental ResultsDecision-based Adversarial AttacksDecision-based Adversarial AttacksDecision-based Adversarial AttacksMy Research Team and CollaboratorsSlide Number 33

security for machine learning systems · computer architectureand robust energy-efficient...

Documents