privacy preserving identi cation using ... -...

Privacy Preserving Identification Using Sparse Approximation with Ambiguization 1 / 34

Privacy Preserving Identification UsingSparse Approximation with Ambiguization

Behrooz Razeghi, Slava Voloshynovskiy, Dimche Kostadinovand Olga Taran

Stochastic Information Processing GroupUniversity of Geneva

Switzerland

December 2017

Behrooz Razeghi Stochastic Information Processing (SIP) Group, University of Geneva, Switzerland 1 / 34


Outline

Introduction

Proposed FrameworkMain IdeaSparse Data RepresentationAmbiguizationPrivacy-Preserving Identification

Results



Introduction

IntroductionPrivacy-preserving content identification

Biometrics

Physical object recognition and security

Medical/clinical applications

Privacy-sensitive multimedia records



Introduction

IntroductionPrivacy-preserving content identification

Biometrics

Physical object recognition and security

Recent Trends

Big Data & Distributed Applications

Services on outsourced

cloud-based systems

Medical/clinical applications

Privacy-sensitive multimedia records



Introduction

IntroductionProblem Formulation

Goal of privacy protection in outsourced services

y

x, yϕ(y)

ϕ (X)

[ϕ (x (1)) , ..., ϕ (x (M))]



Introduction

IntroductionHow do we receive a feature vector?

Encoding & Feature Aggregation

I Local/Global Descriptors

I BoW

I VLAD

I Fisher Vectors

I Triangulation Embedding

I · · ·

I Average Pooling

I Max Pooling

I Democratic Aggregation

1

N

I x(m) ∈ RN

Feature Vector

Layer 1 Layer 2 Layer L

Convolutional Neural Network

1

N

I x(m) ∈ RN

Feature Vector



Introduction

Introductionstate-of-the-art

Cryptographic Methods - Homomorphic Encryption

• Main Idea: Similarity search in the encrypted domain

• Brute force identification =⇒ huge complexity

Robust Hashing - a single hash from the whole content /local descriptors / last layer of CNN

• Main Idea: x −→ (011011100110) and believed non-invertability

• Loss in performance due to binarization

• Unauthorized database clustering

Group Testing / Memory Vectors

• Main Idea: Group testing by measuring the proximity to the group

representative

• Group representatives (memory vectors) should be stored in memory



Introduction


Universal Quantization

• Main Idea: projection with the dimension reduction and periodic

quantization

• Binary quantization: in the region of low projected magnitudes -high Pb

• Ambiguization due to periodization of quantizer - no possibility torecover data even for the authorized users

• Server can still can cluster data - privacy leakages

• Information preservation in general - no link to R(d) and recovery is

demonstrated so far

a = ψ (Wx)

ti = [Wx]i

+1

−1

t

ψ(t)



Introduction


Proposed approach: 3 key elements

• Sparsification• Ambiguization• Search / Identification

• Advantages:

• Fast search / memory efficient• Difficult to accurately reconstruct from probe• Server cannot reveal a structure of the database

• Main concerns addressed in our study:

• Performance• Memory (database) / complexity (identification)• Privacy-preserving with respect to:

database Aprobe y



Proposed Framework

Main Idea

Part 1:Sparse Data Representation



Proposed Framework

Main Idea

SparsificationMain Idea

1

N

I x(m) ∈ RN

I x(m) ∼ p (x)

Encoder

1

L ≥ N

I a(m) ∈ −1, 0,+1L

I ‖a(m)‖0 ≤ Sx

I Rate : R =1

Llog2

((L

Sx

)2Sx

)

Decoder

1

N

I x(m) ∈ RN

I Distortion : 1N ‖x(m)− x(m)‖22 ≤ D



Proposed Framework

Main Idea

SparsificationMain Idea

1

N

I x(m) ∈ RN

I x(m) ∼ p (x)

Encoder

1

L ≥ N

I a(m) ∈ −1, 0,+1L

I ‖a(m)‖0 ≤ Sx

I Rate : R =1

Llog2

((L

Sx

)2Sx

)

Decoder

1

N

I x(m) ∈ RN

I Distortion : 1N ‖x(m)− x(m)‖22 ≤ D

Preservation of Information

Rate-Distortion Function

0 0.2 0.4 0.6 0.8 10

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9



Proposed Framework

Main Idea

Part 2:Ambiguization



Proposed Framework

Main Idea

AmbiguizationMain Idea

1

L

I a(m) ∈ −1, 0,+1L

I ‖a(m)‖0 ≤ Sx

Ambiguization

addition ofnoisy samples

( )

1

L

I a(m)⊕

n

I Public Domain

Decoder

1

N

I x(m) ∈ RN

I ‖x(m)− x(m)‖22 →'2Nσ2x

I Prevent reconstruction from a(m) and from probe q

I Preclude server from discovering the structure of the database A



Proposed Framework

Main Idea

AmbiguizationMain Idea

1

L

I a(m) ∈ −1, 0,+1L

I ‖a(m)‖0 ≤ Sx

Ambiguization

addition ofnoisy samples

( )

1

L

I Public Domain

I a(m)⊕

n

Decoder

Even knowingtransform

1

N

I x(m) ∈ RN

I ‖x(m)− x(m)‖22 →'2Nσ2x

I Prevent reconstruction from a(m)⊕

n and from probe y

I Preclude server from discovering the structure of the database A



Proposed Framework

Main Idea

Part 3:Privacy-Preserving Identification



Proposed Framework

Main Idea

Privacy-Preserving Identification: Private SearchMain Idea: User discloses his probe completely

1

N

I y ∈ RN

1

L

I b ∈ −1, 0,+1L

I ‖b‖0 ≤ Sy

Encoder



Proposed Framework

Main Idea

Privacy-Preserving Identification: Private SearchMain Idea: User discloses his probe completely

1

N

I y ∈ RN

1

L

I b ∈ −1, 0,+1L

I ‖b‖0 ≤ Sy

Encoder

Public Sparse Storage

x(1) · · · x(M)

1

...

L

+100−10...

· · ·

· · ·

0−100+1...

ANN Decoder

List

L (y)



Proposed Framework

Main Idea

Privacy-Preserving Identification: Public SearchMain Idea: User sends only positions of interest

1

N

I y ∈ RN

1

L

I b ∈ −1, 0,+1L

I ‖b‖0 ≤ Sy

I Add Snq random positions

Encoder

Public List

Refined Encoding

List

L2 (y)



Proposed Framework

Main Idea


1

N

I y ∈ RN

1

L

I b ∈ −1, 0,+1L

I ‖b‖0 ≤ Sy


Encoder


x(1) · · · x(M)

1

...

L

+100−10...

· · ·

· · ·

0−100

+1...

ANN Decoderpy

Sy + Snq positions

Public List

L1 (y)

Refined Encoding

List

L2 (y)



Proposed Framework

Main Idea


1

N

I y ∈ RN

1

L

I b ∈ −1, 0,+1L

I ‖b‖0 ≤ Sy


sign (b)

Encoder


x(1) · · · x(M)

1

...

L

+100−10...

· · ·

· · ·

0−100

+1...

ANN Decoderpy

Sy + Snq positions

Public List

L1 (y)

Refined Decoder

Private List

L2 (y)



Proposed Framework

Main Idea

Main idea behind the proposed solutionOwner

X = x (1) , ...,x (m) , ...,x (M)

X ∈ XN×M

Encoder

a(m)=ϕx (x(m))︸︷︷︸

a(m)

⊕n

Public Storage

A = a (1) , ..., a (m) , ..., a (M)

A ∈ AL×M

x

y = x (m) + z

y = x

py (positions)

(Pubic Decoding)

List

L1 (y)

Refined Encoding

List

L2 (y)



Proposed Framework

Main Idea


X = x (1) , ...,x (m) , ...,x (M)

X ∈ XN×M

Encoder

a(m)=ϕx (x(m))︸︷︷︸

a(m)

⊕n

Encoder

b = ϕy (y)

Public Storage

A = a (1) , ..., a (m) , ..., a (M)

A ∈ AL×M

p(y (m) | x (m))

x

Probe

Data User

y = x (m) + z

y = x

ANN Decoding

d (px (m) ,py) ≤ γL

1 ≤ m ≤M

py (positions)

(Pubic Decoding)

List

L1 (y)

Refined Encoding

List

L2 (y)



Proposed Framework

Main Idea


X = x (1) , ...,x (m) , ...,x (M)

X ∈ XN×M

Encoder

a(m)=ϕx (x(m))︸︷︷︸

a(m)

⊕n

Encoder

b = ϕy (y)

Public Storage

A = a (1) , ..., a (m) , ..., a (M)

A ∈ AL×M

p(y (m) | x (m))

x

Probe

Data User

y = x (m) + z

y = x

ANN Decoding

d (px (m) ,py) ≤ γL

1 ≤ m ≤M

py (positions)

(Pubic Decoding)

List

L1 (y)

Refined Decoding

dsupp(b) (a (m)⊕

n,b) ≤ ζL

m ∈ L1 (y)

List

L2 (y)



Proposed Framework

Sparse Data Representation

Sparsifying TransformA Schematic Idea

1

N

1

L

1

L

ψ(Wx(m))

+1

−1

λx−λx

Linear Mapping Element-wise Non-linearity

x (m) ∈ RN Wx (m) ∈ RL a (m) ∈ −1, 0,+1L

ϕ(.)

W ψ (.)



Proposed Framework


Sparsifying TransformGeneral Problem Formulation

1

N

x(m) ∈ RN

Encoder

1

L ≥ N

a(m) ∈ −1, 0,+1L

Decoder

1

N

x(m) ∈ RN

Given

W

Random Learned from data

Encoder:

a (m) = ψ (Wx (m))

Decoder:

x (m) = W†a (m)



Proposed Framework


Encoder: as a projection problem (for a fixed W)

a(m)= arg mina(m)∈AL

‖Wx(m)−a(m)‖22 + βΩ (a(m)),∀m ∈ [M ]

- W ∈ RL×N , x(m) ∈ RN , a(m) ∈ RL

- Closed-form solution for: Ω (.) = ‖.‖0 and Ω (.) = ‖.‖1

ti

ψ(ti)

Hard-thresholding operator

Ω (.) = ‖.‖0

λx−λx ti

ψ(ti)

Soft-thresholding operator

Ω (.) = ‖.‖1

λx−λxti = [Wx]i

a (m) = ψ (Wx (m))



Proposed Framework


Encoder: Extra constraint on the alphabet

a (m) = ψ (Wx (m))

s.t. a (m) ∈ −1, 0,+1

ti

ψ(ti)

λx−λx ti

ψ(ti)

λx−λx

+1

−1

ti = [Wx]i

Sparse Ternary Code



Proposed Framework


Encoder: Extra constraint on the alphabet

a (m) = ψ (Wx (m))

s.t. a (m) ∈ −1, 0,+1

ti

ψ(ti)

λx−λx ti

ψ(ti)

λx−λx

+1

−1

ti = [Wx]i

Sparse Ternary Code

Remark:

Binary hashing (like LSH) is the special case of our ψ(.) for λx = 0.

+1

−1

ti

ψ(ti)

λx = 0



Proposed Framework


Comparison of Three Encoding Schemes

Binary Hashing

+1

−1

t

ψ(t)

Quantized Embeddings

+1

−1

t

ψ(t)

Sparse Ternary Coding

+1

−1

t

ψ(t)



Proposed Framework


Learning Sparsifying TransformGeneral Formulation: joint learning(

W, A)

= arg min(W,A)

‖WX−A‖2F + βWΩW (W) + βAΩA(A)

I Sparse Coding Step (Fixed W):

A = arg minA‖WX−A‖2F + βAΩA (A)

a (m) = ψ (Wx (m))

I Transform Update Step (Fixed A):

W = arg minW‖WX−A‖2F + βWΩW (W)

Linear Regression : W = AXT(XXT + βW IN

)−1

(with quadratic regularizer)



Proposed Framework

Ambiguization

Ambiguization SchemeMain Idea

Add noise to non-zero components of sparse representation

0 20 40 60 80 100 120-1.5

-1

-0.5

0

0.5

1

1.5



Proposed Framework

Privacy-Preserving Identification

Desired property of mapping schemeDistance preservation in the desired radius

Privacy

Preserving Region

y εN

x

εN

εN

‖x− y‖2

‖ϕ(x)− ϕ(y)‖2

Presereved Distances

Uniform Distances



Results

Impact of Ambiguization at Server Side

Goal: The server should not distinguish distances ‖ (ψ (Wx)⊕

n)− ψ (Wy) ‖2

Distances are computed in the full length.

0 2 4 6 80

2

4

6

8

10

12

14 ψ (Wx)



Results



n)− ψ (Wy) ‖2


0 2 4 6 80

2

4

6

8

10

12

14 ψ (Wx)

ψ (Wx)⊕

n



Results



n)− ψ (Wy) ‖2


0 2 4 6 80

2

4

6

8

10

12

14 ψ (Wx)

ψ (Wx)⊕

n

Server Side



Results

Impact of Ambiguization at Client Side

Goal: The client should distinguish distances(‖ (ψ (Wx)

⊕n)− ψ (Wy) ‖2

)supp

0 1 2 3 4 5 6 7 8 90

2

4

6

8

10

12

14 Owner’s Data: ψ (Wx)

Server’s Data: ψ (Wx)⊕

n

Client’s Data: ψ (Wy)

Distances are computed in the non-zero components of probe.



Results

Reconstruction: Authorized User I x = W†ax : i.i.d. Gaussian, with each sample Xn ∼ N (0, 1), N

L = 1Sx : Sparsity Level

0 0.2 0.4 0.6 0.8 10

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1



Results

Reconstruction: Unauthorized User I x = W† (a⊕

n)

0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 10

0.2

0.4

0.6

0.8

1

1.2

1.4

1.6

1.8

2

Half Ambiguization: Sns = 12(L− Sx) Full Ambiguization: Sns = (L− Sx)



Results

ClusteringGenerate Structured Data

I Generate:

- Four 512-dimensional i.i.d. vectors with distribution N (0,1)- 1000 512-dimensional i.i.d. vectors with distribution N (0,0.1)

I Add each 250 (out of 1000) low variance vectors to the fourhigh variance ones



Results

Clustering

Goal: Hide structure of database

Original Domain Public Domain

ψ (Wx (m))⊕

n



Results

ClusteringIntroduce Measure for Evaluation

Define:

I αx = SxL

, Sx : Sparsity level

Denote:

I Pintra : PDF of ‘intra-cluster’ distances

I Pinter : PDF of ‘inter-cluster’ distances

Define:

I P1 = αx Pintra + (1− αx)Pinter, 0 ≤ αx ≤ 1

Denote:

I P2 ∼ N(µ2, σ2

2

), fit to P1

Define:

I Privacy Leak Measure:

D (P1‖P2) = αxD (Pintra‖P2) + (1− αx)D (Pinter‖P2)

= EP1

[log

P1

P2

]



Results

ClusteringIntroduce Measure for Evaluation

Define:

I αx = SxL

, Sx : Sparsity level

Denote:

I Pintra : PDF of ‘intra-cluster’ distances

I Pinter : PDF of ‘inter-cluster’ distances

Define:

I P1 = αx Pintra + (1− αx)Pinter, 0 ≤ αx ≤ 1

Denote:

I P2 ∼ N(µ2, σ2

2

), fit to P1

Define:

I Privacy Leak Measure:

D (P1‖P2) = αxD (Pintra‖P2) + (1− αx)D (Pinter‖P2)

= EP1

[log

P1

P2

]

PDF

P1

intra inter

P2

D (P1‖P2) ↑

Clear distinguishability based on inter&intra-distances

PDF

P1

P2

D (P1‖P2) → 0

Not distinguishable



Results

Clustering: How much ambiguization should be added tohave indistinguishability for the server?Evaluation of Our Scheme: αx = Sx

L, βx =

SnsL

, Sns : # of noise components for theserver

0 5 10 15 20 25 30 35Distances

0

0.5

1

1.5

Est

imate

dPD

F

,x : 0:01

-x = 0-x = 0:03125-x = 0:125-x = 0:5-x = 1! ,x

D(P1kP2) = 0:1312

D(P1kP2) = 1:4778

D(P1kP2) = 0:0088

D(P1kP2) = 0:0132

0 5 10 15 20 25 30 35Distances

0

0.5

1

1.5

Est

imate

dPD

F

,x : 0:02

-x = 0-x = 0:03125-x = 0:125-x = 0:5-x = 1! ,x

D(P1kP2) = 0:0098

D(P1kP2) = 0:0321

D(P1kP2) = 9:5504 D(P1kP2) = 1:5719

0 5 10 15 20 25 30 35Distances

0

0.5

1

1.5

Est

imate

dPD

F

,x : 0:045

-x = 0-x = 0:03125-x = 0:125-x = 0:5-x = 1! ,x

D(P1kP2) = 0:5390

D(P1kP2) = 0:0363

D(P1kP2) = 17:4059 D(P1kP2) = 7:7313

0 5 10 15 20 25 30 35Distances

0

0.5

1

1.5Est

imate

dPD

F

,x : 0:09

-x = 0-x = 0:03125-x = 0:125-x = 0:5-x = 1! ,x

D(P1kP2) = 0:3152

D(P1kP2) = 3:4178

D(P1kP2) = 17:4405

D(P1kP2) = 12:4624



Results

Conclusions:

Preserve distances up to the desired radius

Ensure the reconstruction of data for authorized users

Preclude the curious server to cluster or reconstruct thesamples in the database

Public decoding scheme



Results


privacy preserving identi cation using ... -...

Documents