Regret Minimizing Audits:A Learning-theoretic Basis

for Privacy Protection

Jeremiah Blocki, Nicolas Christin, Anupam Datta, Arunesh Sinha

Carnegie Mellon University

2

Motivation

Goal: treatmentRigid access control hinders treatmentPermissive access control ⇒ privacy

violations

Breach

3

A real problem

4

AuditsAudits: one way to address the

problem◦Permissive access control

If in doubt allow access◦Log the accesses◦Human auditors review the accesses

later and find violations

Adhoc approaches in practice◦FairWarning audit tool implements

simple heuristics, e.g., flag all celebrity access

5

DesiderataPrincipled study of the audit

process◦A model for audit process◦Properties of the audit mechanism◦Audit mechanism which provably

satisfies the property

6

Auditing ChallengesOrganization’s economic tradeoff

Employee’s incentives unknown

How to optimally allocate budget for auditing, with no knowledge about adversary incentives?

Reputationloss

Auditcost

Audit Algorithm by Example

Overview

Audit Model

Low Regret

Algorith

m

Auditing budget: $3000/ cycle

Cost for one inspection: $100

Only 30 inspections per cycle Auditor

100 accesses

30 accesses

70 accesses

Access divided into 2 types

Loss from 1 violation(internal, external)

$500, $1000

$250, $500

8

Audit Algorithm ChoicesOnly 30

inspections

0 10 20 30

30 20 10 0

Consider 4 possible allocations of the

available 30 inspections

1.0 1.0 1.0 1.0Weights

Choose allocation probabilistically based on weights

Overview

Audit Model

Low Regret

Algorith

m

9

No. ofAccess

Audit Algorithm Run0 10 20 30

30 20 10 0

0.5 0.5 2.0 1.5Updated weights

ObservedLoss

$2000 $1500 $1000$100

0

$750 $1250 $1250$150

0

Learn from experience: weights updated using observed and estimated loss

2

4

ActualViolati

on

Ext.Caught

Int.Caught

1 1

12

30

70

Overview

Audit Model

Low Regret

Algorith

m

EstimatedLoss

10

Main Contributions A game model for the audit

process Defining a desirable property of

audit mechanisms, namely low regret

An efficient audit mechanism RMA that provably achieves low regret

o Better bound on regret than existing algorithms that achieve low regret

Overview

Audit Model

Low Regret

Algorith

m

11

Repeated Game ModelGame model

The interaction repeats for each audit cycle (typically called rounds of repeated game)

Typical actions in one round ◦Emp action: (access, violate) = ([30,70],

[2,4])◦Org action: inspection = ([10,20])

InspectAccess , Violate

One auditcycle (round)

Imperfection

Overview

Audit Model

Low Regret

Algorith

m

12

Game PayoffsOrganization’s payoff

◦ Audit cost depends on the number of inspections

◦ Reputation loss depends on the number of

violations caught

Employee’s payoff unknown

Reputationloss

Auditcost

Overview

Audit Model

Low Regret

Algorith

m

13

Regret Intuition

Is it possible to audit as well as the best strategy in hindsight ?

0 10 20 30

30 20 10 0

Overview

Audit Model

Low Regret

Algorith

m

14

Regret by Example

$5 $6

$0 $5

1 2

3,1

3, 2

Payoff of Org only

Total regret(s, s1) = (–5) – (–6) = 1regret(s, s1) = 1/2

Strategy: outputs an actionfor every round

Emp

Org

Players

• Emp• Org: s

Round 1

• 3,1• 2 ($6 )

Round 2

• 3,2• 1 ($0)

Total Payoff

• Unknown•$6

Org :s1 1 ($5) 1 ($0) $5

Overview

Audit Model

Low Regret

Algorith

m

15

Meaning of RegretLow regret of s w.r.t. s1 means s

performs as well as s1Desirable property of an audit

mechanism◦Low regret w.r.t all strategies in a given

set of strategies◦regret → 0 as T → ∞

Overview

Audit Model

Low Regret

Algorith

m

16

Regret minimization Multiplicative weight update

(MWU)◦is a standard algorithm that achieves

low regret w.r.t. to all strategies in a given set

The regret bound of MWU is

◦N: number of strategies in the given set

◦T: number of rounds of the game◦All payoffs scaled to lie in [0,1]

Overview

Audit Model

Low Regret

Algorith

m

17

Why not MWU? Imperfect information

◦ Org never learns the true action (violation) of the employee

◦ RMA regret bound: O((ln N)/T) Best known bounds [ACFS03] : O((N1/3 ln N)/T1/3) Idea: estimate the payoff that would have been

received

Sleeping strategies: unavailable strategies◦ Some inspections unavailable due to

budgetary constraints◦ We use techniques from [BM05]

[ACFS03] P. Auer, N. Cesa-Bianchi, Y. Freund, R. Schapire, “The nonstochastic multiarmed bandit problem,” SIAM Journal on Computing, 2003

[BM05] A. Blum and Y. Mansour, “From external to internal regret,”in COLT 2005

Overview

Audit Model

Low Regret

Algorith

m

18

Regret Minimizing Audits (RMA)

New audit cycle starts. Find AWAKE

Pick s in AWAKE with probability Dt(s) w∝ s

Update weight* of strategies s in AWAKE

Estimate payoff vector Pay using Pay(s)

Violation caught; obtain payoff Pay(s)

ws = 1 for all strategies s

*

Overview

Audit Model

Low Regret

Algorith

m

19

Guarantees of RMAWith probability RMA

achieves the regret bound

◦N is the set of strategies◦T is the number of rounds◦All payoffs scaled to lie in [0,1]

Overview

Audit Model

Low Regret

Algorith

m

20

Related Work Authorization proof recorded in audit log

[Vaughan et al. 2008] Analyze audit logs to detect and resolve access

control policy misconfigurations [Bauer et al. 2008]

Mechanically checkable complaince proof constructed using evidence from audit logs [Cederquist et al. 2007]

Mechanically checking policy compliance over incomplete audit logs [Garg et al. 2011]

Take Away Message

Future Work◦Evaluation over real hospital audit

logs◦Analyze performance with more

complex adversary models Worst case + rational

Learning technique for effective auditing with imperfect information

22