deep leakage from gradients...dlg only requires the gradients and can reveal pixel-wise accurate...

Deep Leakage from Gradients

Ligeng Zhu Zhijian Liu Song HanMassachusetts Institute of Technology{ligeng, zhijian, songhan}@mit.edu

Abstract

Exchanging gradients is a widely used method in modern multi-node machinelearning system (e.g., distributed training, collaborative learning). For a long time,people believed that gradients are safe to share: i.e., the training data will not beleaked by gradients exchange. However, we show that it is possible to obtain theprivate training data from the publicly shared gradients. We name this leakage asDeep Leakage from Gradient and empirically validate the effectiveness on bothcomputer vision and natural language processing tasks. Experimental results showthat our attack is much stronger than previous approaches: the recovery is pixel-wise accurate for images and token-wise matching for texts. Thereby we want toraise people’s awareness to rethink the gradient’s safety. We also discuss severalpossible strategies to prevent such deep leakage. Without changes on trainingsetting, the most effective defense method is gradient pruning.

1 Introduction

Distributed training becomes necessary to speedup training on large-scale datasets. In a distributedlearning system, the computation is executed parallely on each worker and synchronized via exchang-ing gradients (both parameter server [15, 23] and all-reduce [3, 30]). The distribution of computationnaturally leads to the splitting of data: Each client has its own the training data and only communicatesgradient during training (says the training set never leaves local machine). It allows to train a modelusing data from multiple sources without centralizing them. This scheme is named as collaborativelearning and widely used when the training set contains private information [18, 20]. For example,multiple hospitals train a model jointly without sharing their patients’ medical data [17, 26].

Distributed training and collaborative learning have been widely used in large scale machine learningtasks. However, does the “gradient sharing” scheme protect the privacy of the training datasetsof each participant? In most scenarios, people assume that gradients are safe to share and willnot expose the training data. Some recent studies show that gradients reveal some properties ofthe training data, for example, property classifier [27] (whether a sample with certain property isin the batch) and using generative adversarial networks to generate pictures that look similar to thetraining images [9, 13, 27]. Here we consider a more challenging case: can we completely steal thetraining data from gradients? Formally, given a machine learning model F () and its weights W , ifwe have the gradients∇w w.r.t a pair of input and label, can we obtain the training data reversely?Conventional wisdom suggests that the answer is no, but we show that this is actually possible.

In this work, we demonstrate Deep Leakage from Gradients (DLG): sharing the gradients can leakprivate training data. We present an optimization algorithm that can obtain both the training inputsand the labels in just few iterations. To perform the attack, we first randomly generate a pair of“dummy” inputs and labels and then perform the usual forward and backward. After deriving thedummy gradients from the dummy data, instead of optimizing model weights as in typical training,we optimize the dummy inputs and labels to minimize the distance between dummy gradients andreal gradients (illustrated in Fig. 2). Matching the gradients makes the dummy data close to the

33rd Conference on Neural Information Processing Systems (NeurIPS 2019), Vancouver, Canada.

Parameter Server�W1

<latexit sha1_base64="K3SAkvXn+lryYWGHf203kFgdJgw=">AAAB8XicbVDLSgNBEOz1GeMr6tHLYBA8hV0RzDGgB48RzAOTEGYnnWTI7Owy0yuEkL/w4kERr/6NN//GSbIHTSxoKKq66e4KEyUt+f63t7a+sbm1ndvJ7+7tHxwWjo7rNk6NwJqIVWyaIbeopMYaSVLYTAzyKFTYCEc3M7/xhMbKWD/QOMFOxAda9qXg5KTH9i0q4qzRDbqFol/y52CrJMhIETJUu4Wvdi8WaYSahOLWtgI/oc6EG5JC4TTfTi0mXIz4AFuOah6h7UzmF0/ZuVN6rB8bV5rYXP09MeGRteModJ0Rp6Fd9mbif14rpX65M5E6SQm1WCzqp4pRzGbvs540KEiNHeHCSHcrE0NuuCAXUt6FECy/vErql6XALwX3V8VKOYsjB6dwBhcQwDVU4A6qUAMBGp7hFd486714797HonXNy2ZO4A+8zx+KjZAh</latexit><latexit sha1_base64="K3SAkvXn+lryYWGHf203kFgdJgw=">AAAB8XicbVDLSgNBEOz1GeMr6tHLYBA8hV0RzDGgB48RzAOTEGYnnWTI7Owy0yuEkL/w4kERr/6NN//GSbIHTSxoKKq66e4KEyUt+f63t7a+sbm1ndvJ7+7tHxwWjo7rNk6NwJqIVWyaIbeopMYaSVLYTAzyKFTYCEc3M7/xhMbKWD/QOMFOxAda9qXg5KTH9i0q4qzRDbqFol/y52CrJMhIETJUu4Wvdi8WaYSahOLWtgI/oc6EG5JC4TTfTi0mXIz4AFuOah6h7UzmF0/ZuVN6rB8bV5rYXP09MeGRteModJ0Rp6Fd9mbif14rpX65M5E6SQm1WCzqp4pRzGbvs540KEiNHeHCSHcrE0NuuCAXUt6FECy/vErql6XALwX3V8VKOYsjB6dwBhcQwDVU4A6qUAMBGp7hFd486714797HonXNy2ZO4A+8zx+KjZAh</latexit><latexit sha1_base64="K3SAkvXn+lryYWGHf203kFgdJgw=">AAAB8XicbVDLSgNBEOz1GeMr6tHLYBA8hV0RzDGgB48RzAOTEGYnnWTI7Owy0yuEkL/w4kERr/6NN//GSbIHTSxoKKq66e4KEyUt+f63t7a+sbm1ndvJ7+7tHxwWjo7rNk6NwJqIVWyaIbeopMYaSVLYTAzyKFTYCEc3M7/xhMbKWD/QOMFOxAda9qXg5KTH9i0q4qzRDbqFol/y52CrJMhIETJUu4Wvdi8WaYSahOLWtgI/oc6EG5JC4TTfTi0mXIz4AFuOah6h7UzmF0/ZuVN6rB8bV5rYXP09MeGRteModJ0Rp6Fd9mbif14rpX65M5E6SQm1WCzqp4pRzGbvs540KEiNHeHCSHcrE0NuuCAXUt6FECy/vErql6XALwX3V8VKOYsjB6dwBhcQwDVU4A6qUAMBGp7hFd486714797HonXNy2ZO4A+8zx+KjZAh</latexit><latexit sha1_base64="K3SAkvXn+lryYWGHf203kFgdJgw=">AAAB8XicbVDLSgNBEOz1GeMr6tHLYBA8hV0RzDGgB48RzAOTEGYnnWTI7Owy0yuEkL/w4kERr/6NN//GSbIHTSxoKKq66e4KEyUt+f63t7a+sbm1ndvJ7+7tHxwWjo7rNk6NwJqIVWyaIbeopMYaSVLYTAzyKFTYCEc3M7/xhMbKWD/QOMFOxAda9qXg5KTH9i0q4qzRDbqFol/y52CrJMhIETJUu4Wvdi8WaYSahOLWtgI/oc6EG5JC4TTfTi0mXIz4AFuOah6h7UzmF0/ZuVN6rB8bV5rYXP09MeGRteModJ0Rp6Fd9mbif14rpX65M5E6SQm1WCzqp4pRzGbvs540KEiNHeHCSHcrE0NuuCAXUt6FECy/vErql6XALwX3V8VKOYsjB6dwBhcQwDVU4A6qUAMBGp7hFd486714797HonXNy2ZO4A+8zx+KjZAh</latexit>

�W1<latexit sha1_base64="K3SAkvXn+lryYWGHf203kFgdJgw=">AAAB8XicbVDLSgNBEOz1GeMr6tHLYBA8hV0RzDGgB48RzAOTEGYnnWTI7Owy0yuEkL/w4kERr/6NN//GSbIHTSxoKKq66e4KEyUt+f63t7a+sbm1ndvJ7+7tHxwWjo7rNk6NwJqIVWyaIbeopMYaSVLYTAzyKFTYCEc3M7/xhMbKWD/QOMFOxAda9qXg5KTH9i0q4qzRDbqFol/y52CrJMhIETJUu4Wvdi8WaYSahOLWtgI/oc6EG5JC4TTfTi0mXIz4AFuOah6h7UzmF0/ZuVN6rB8bV5rYXP09MeGRteModJ0Rp6Fd9mbif14rpX65M5E6SQm1WCzqp4pRzGbvs540KEiNHeHCSHcrE0NuuCAXUt6FECy/vErql6XALwX3V8VKOYsjB6dwBhcQwDVU4A6qUAMBGp7hFd486714797HonXNy2ZO4A+8zx+KjZAh</latexit><latexit sha1_base64="K3SAkvXn+lryYWGHf203kFgdJgw=">AAAB8XicbVDLSgNBEOz1GeMr6tHLYBA8hV0RzDGgB48RzAOTEGYnnWTI7Owy0yuEkL/w4kERr/6NN//GSbIHTSxoKKq66e4KEyUt+f63t7a+sbm1ndvJ7+7tHxwWjo7rNk6NwJqIVWyaIbeopMYaSVLYTAzyKFTYCEc3M7/xhMbKWD/QOMFOxAda9qXg5KTH9i0q4qzRDbqFol/y52CrJMhIETJUu4Wvdi8WaYSahOLWtgI/oc6EG5JC4TTfTi0mXIz4AFuOah6h7UzmF0/ZuVN6rB8bV5rYXP09MeGRteModJ0Rp6Fd9mbif14rpX65M5E6SQm1WCzqp4pRzGbvs540KEiNHeHCSHcrE0NuuCAXUt6FECy/vErql6XALwX3V8VKOYsjB6dwBhcQwDVU4A6qUAMBGp7hFd486714797HonXNy2ZO4A+8zx+KjZAh</latexit><latexit sha1_base64="K3SAkvXn+lryYWGHf203kFgdJgw=">AAAB8XicbVDLSgNBEOz1GeMr6tHLYBA8hV0RzDGgB48RzAOTEGYnnWTI7Owy0yuEkL/w4kERr/6NN//GSbIHTSxoKKq66e4KEyUt+f63t7a+sbm1ndvJ7+7tHxwWjo7rNk6NwJqIVWyaIbeopMYaSVLYTAzyKFTYCEc3M7/xhMbKWD/QOMFOxAda9qXg5KTH9i0q4qzRDbqFol/y52CrJMhIETJUu4Wvdi8WaYSahOLWtgI/oc6EG5JC4TTfTi0mXIz4AFuOah6h7UzmF0/ZuVN6rB8bV5rYXP09MeGRteModJ0Rp6Fd9mbif14rpX65M5E6SQm1WCzqp4pRzGbvs540KEiNHeHCSHcrE0NuuCAXUt6FECy/vErql6XALwX3V8VKOYsjB6dwBhcQwDVU4A6qUAMBGp7hFd486714797HonXNy2ZO4A+8zx+KjZAh</latexit><latexit sha1_base64="K3SAkvXn+lryYWGHf203kFgdJgw=">AAAB8XicbVDLSgNBEOz1GeMr6tHLYBA8hV0RzDGgB48RzAOTEGYnnWTI7Owy0yuEkL/w4kERr/6NN//GSbIHTSxoKKq66e4KEyUt+f63t7a+sbm1ndvJ7+7tHxwWjo7rNk6NwJqIVWyaIbeopMYaSVLYTAzyKFTYCEc3M7/xhMbKWD/QOMFOxAda9qXg5KTH9i0q4qzRDbqFol/y52CrJMhIETJUu4Wvdi8WaYSahOLWtgI/oc6EG5JC4TTfTi0mXIz4AFuOah6h7UzmF0/ZuVN6rB8bV5rYXP09MeGRteModJ0Rp6Fd9mbif14rpX65M5E6SQm1WCzqp4pRzGbvs540KEiNHeHCSHcrE0NuuCAXUt6FECy/vErql6XALwX3V8VKOYsjB6dwBhcQwDVU4A6qUAMBGp7hFd486714797HonXNy2ZO4A+8zx+KjZAh</latexit>

(1) Distributed training with a centralized server (2) Distributed training without a centralized server


Parameter Server



(2) Distributed training without a centralized server

Parameter Server

…

Flower Cat

…

Flower Cat

LeakrW

<latexit sha1_base64="5IzzVUcy5d0XJ5bDlcrRTgLMUCA=">AAAB73icbVBNS8NAEJ3Ur1q/qh69LBbBU0lEsMeCF48V7Ae0oUy2m3bpZhN3N0IJ/RNePCji1b/jzX/jNs1BWx8MPN6bYWZekAiujet+O6WNza3tnfJuZW//4PCoenzS0XGqKGvTWMSqF6BmgkvWNtwI1ksUwygQrBtMbxd+94kpzWP5YGYJ8yMcSx5yisZKvYHEQCDpDqs1t+7mIOvEK0gNCrSG1a/BKKZpxKShArXue25i/AyV4VSweWWQapYgneKY9S2VGDHtZ/m9c3JhlREJY2VLGpKrvycyjLSeRYHtjNBM9Kq3EP/z+qkJG37GZZIaJulyUZgKYmKyeJ6MuGLUiJklSBW3txI6QYXU2IgqNgRv9eV10rmqe27du7+uNRtFHGU4g3O4BA9uoAl30II2UBDwDK/w5jw6L86787FsLTnFzCn8gfP5A393j5E=</latexit><latexit sha1_base64="5IzzVUcy5d0XJ5bDlcrRTgLMUCA=">AAAB73icbVBNS8NAEJ3Ur1q/qh69LBbBU0lEsMeCF48V7Ae0oUy2m3bpZhN3N0IJ/RNePCji1b/jzX/jNs1BWx8MPN6bYWZekAiujet+O6WNza3tnfJuZW//4PCoenzS0XGqKGvTWMSqF6BmgkvWNtwI1ksUwygQrBtMbxd+94kpzWP5YGYJ8yMcSx5yisZKvYHEQCDpDqs1t+7mIOvEK0gNCrSG1a/BKKZpxKShArXue25i/AyV4VSweWWQapYgneKY9S2VGDHtZ/m9c3JhlREJY2VLGpKrvycyjLSeRYHtjNBM9Kq3EP/z+qkJG37GZZIaJulyUZgKYmKyeJ6MuGLUiJklSBW3txI6QYXU2IgqNgRv9eV10rmqe27du7+uNRtFHGU4g3O4BA9uoAl30II2UBDwDK/w5jw6L86787FsLTnFzCn8gfP5A393j5E=</latexit><latexit sha1_base64="5IzzVUcy5d0XJ5bDlcrRTgLMUCA=">AAAB73icbVBNS8NAEJ3Ur1q/qh69LBbBU0lEsMeCF48V7Ae0oUy2m3bpZhN3N0IJ/RNePCji1b/jzX/jNs1BWx8MPN6bYWZekAiujet+O6WNza3tnfJuZW//4PCoenzS0XGqKGvTWMSqF6BmgkvWNtwI1ksUwygQrBtMbxd+94kpzWP5YGYJ8yMcSx5yisZKvYHEQCDpDqs1t+7mIOvEK0gNCrSG1a/BKKZpxKShArXue25i/AyV4VSweWWQapYgneKY9S2VGDHtZ/m9c3JhlREJY2VLGpKrvycyjLSeRYHtjNBM9Kq3EP/z+qkJG37GZZIaJulyUZgKYmKyeJ6MuGLUiJklSBW3txI6QYXU2IgqNgRv9eV10rmqe27du7+uNRtFHGU4g3O4BA9uoAl30II2UBDwDK/w5jw6L86787FsLTnFzCn8gfP5A393j5E=</latexit><latexit sha1_base64="5IzzVUcy5d0XJ5bDlcrRTgLMUCA=">AAAB73icbVBNS8NAEJ3Ur1q/qh69LBbBU0lEsMeCF48V7Ae0oUy2m3bpZhN3N0IJ/RNePCji1b/jzX/jNs1BWx8MPN6bYWZekAiujet+O6WNza3tnfJuZW//4PCoenzS0XGqKGvTWMSqF6BmgkvWNtwI1ksUwygQrBtMbxd+94kpzWP5YGYJ8yMcSx5yisZKvYHEQCDpDqs1t+7mIOvEK0gNCrSG1a/BKKZpxKShArXue25i/AyV4VSweWWQapYgneKY9S2VGDHtZ/m9c3JhlREJY2VLGpKrvycyjLSeRYHtjNBM9Kq3EP/z+qkJG37GZZIaJulyUZgKYmKyeJ6MuGLUiJklSBW3txI6QYXU2IgqNgRv9eV10rmqe27du7+uNRtFHGU4g3O4BA9uoAl30II2UBDwDK/w5jw6L86787FsLTnFzCn8gfP5A393j5E=</latexit>

rW<latexit sha1_base64="5IzzVUcy5d0XJ5bDlcrRTgLMUCA=">AAAB73icbVBNS8NAEJ3Ur1q/qh69LBbBU0lEsMeCF48V7Ae0oUy2m3bpZhN3N0IJ/RNePCji1b/jzX/jNs1BWx8MPN6bYWZekAiujet+O6WNza3tnfJuZW//4PCoenzS0XGqKGvTWMSqF6BmgkvWNtwI1ksUwygQrBtMbxd+94kpzWP5YGYJ8yMcSx5yisZKvYHEQCDpDqs1t+7mIOvEK0gNCrSG1a/BKKZpxKShArXue25i/AyV4VSweWWQapYgneKY9S2VGDHtZ/m9c3JhlREJY2VLGpKrvycyjLSeRYHtjNBM9Kq3EP/z+qkJG37GZZIaJulyUZgKYmKyeJ6MuGLUiJklSBW3txI6QYXU2IgqNgRv9eV10rmqe27du7+uNRtFHGU4g3O4BA9uoAl30II2UBDwDK/w5jw6L86787FsLTnFzCn8gfP5A393j5E=</latexit><latexit sha1_base64="5IzzVUcy5d0XJ5bDlcrRTgLMUCA=">AAAB73icbVBNS8NAEJ3Ur1q/qh69LBbBU0lEsMeCF48V7Ae0oUy2m3bpZhN3N0IJ/RNePCji1b/jzX/jNs1BWx8MPN6bYWZekAiujet+O6WNza3tnfJuZW//4PCoenzS0XGqKGvTWMSqF6BmgkvWNtwI1ksUwygQrBtMbxd+94kpzWP5YGYJ8yMcSx5yisZKvYHEQCDpDqs1t+7mIOvEK0gNCrSG1a/BKKZpxKShArXue25i/AyV4VSweWWQapYgneKY9S2VGDHtZ/m9c3JhlREJY2VLGpKrvycyjLSeRYHtjNBM9Kq3EP/z+qkJG37GZZIaJulyUZgKYmKyeJ6MuGLUiJklSBW3txI6QYXU2IgqNgRv9eV10rmqe27du7+uNRtFHGU4g3O4BA9uoAl30II2UBDwDK/w5jw6L86787FsLTnFzCn8gfP5A393j5E=</latexit><latexit sha1_base64="5IzzVUcy5d0XJ5bDlcrRTgLMUCA=">AAAB73icbVBNS8NAEJ3Ur1q/qh69LBbBU0lEsMeCF48V7Ae0oUy2m3bpZhN3N0IJ/RNePCji1b/jzX/jNs1BWx8MPN6bYWZekAiujet+O6WNza3tnfJuZW//4PCoenzS0XGqKGvTWMSqF6BmgkvWNtwI1ksUwygQrBtMbxd+94kpzWP5YGYJ8yMcSx5yisZKvYHEQCDpDqs1t+7mIOvEK0gNCrSG1a/BKKZpxKShArXue25i/AyV4VSweWWQapYgneKY9S2VGDHtZ/m9c3JhlREJY2VLGpKrvycyjLSeRYHtjNBM9Kq3EP/z+qkJG37GZZIaJulyUZgKYmKyeJ6MuGLUiJklSBW3txI6QYXU2IgqNgRv9eV10rmqe27du7+uNRtFHGU4g3O4BA9uoAl30II2UBDwDK/w5jw6L86787FsLTnFzCn8gfP5A393j5E=</latexit><latexit sha1_base64="5IzzVUcy5d0XJ5bDlcrRTgLMUCA=">AAAB73icbVBNS8NAEJ3Ur1q/qh69LBbBU0lEsMeCF48V7Ae0oUy2m3bpZhN3N0IJ/RNePCji1b/jzX/jNs1BWx8MPN6bYWZekAiujet+O6WNza3tnfJuZW//4PCoenzS0XGqKGvTWMSqF6BmgkvWNtwI1ksUwygQrBtMbxd+94kpzWP5YGYJ8yMcSx5yisZKvYHEQCDpDqs1t+7mIOvEK0gNCrSG1a/BKKZpxKShArXue25i/AyV4VSweWWQapYgneKY9S2VGDHtZ/m9c3JhlREJY2VLGpKrvycyjLSeRYHtjNBM9Kq3EP/z+qkJG37GZZIaJulyUZgKYmKyeJ6MuGLUiJklSBW3txI6QYXU2IgqNgRv9eV10rmqe27du7+uNRtFHGU4g3O4BA9uoAl30II2UBDwDK/w5jw6L86787FsLTnFzCn8gfP5A393j5E=</latexit>

Flower Cat

LeakrW



Flower Cat

(a) Distributed training with a centralized server

Parameter Server�W1

<latexit sha1_base64="K3SAkvXn+lryYWGHf203kFgdJgw=">AAAB8XicbVDLSgNBEOz1GeMr6tHLYBA8hV0RzDGgB48RzAOTEGYnnWTI7Owy0yuEkL/w4kERr/6NN//GSbIHTSxoKKq66e4KEyUt+f63t7a+sbm1ndvJ7+7tHxwWjo7rNk6NwJqIVWyaIbeopMYaSVLYTAzyKFTYCEc3M7/xhMbKWD/QOMFOxAda9qXg5KTH9i0q4qzRDbqFol/y52CrJMhIETJUu4Wvdi8WaYSahOLWtgI/oc6EG5JC4TTfTi0mXIz4AFuOah6h7UzmF0/ZuVN6rB8bV5rYXP09MeGRteModJ0Rp6Fd9mbif14rpX65M5E6SQm1WCzqp4pRzGbvs540KEiNHeHCSHcrE0NuuCAXUt6FECy/vErql6XALwX3V8VKOYsjB6dwBhcQwDVU4A6qUAMBGp7hFd486714797HonXNy2ZO4A+8zx+KjZAh</latexit><latexit sha1_base64="K3SAkvXn+lryYWGHf203kFgdJgw=">AAAB8XicbVDLSgNBEOz1GeMr6tHLYBA8hV0RzDGgB48RzAOTEGYnnWTI7Owy0yuEkL/w4kERr/6NN//GSbIHTSxoKKq66e4KEyUt+f63t7a+sbm1ndvJ7+7tHxwWjo7rNk6NwJqIVWyaIbeopMYaSVLYTAzyKFTYCEc3M7/xhMbKWD/QOMFOxAda9qXg5KTH9i0q4qzRDbqFol/y52CrJMhIETJUu4Wvdi8WaYSahOLWtgI/oc6EG5JC4TTfTi0mXIz4AFuOah6h7UzmF0/ZuVN6rB8bV5rYXP09MeGRteModJ0Rp6Fd9mbif14rpX65M5E6SQm1WCzqp4pRzGbvs540KEiNHeHCSHcrE0NuuCAXUt6FECy/vErql6XALwX3V8VKOYsjB6dwBhcQwDVU4A6qUAMBGp7hFd486714797HonXNy2ZO4A+8zx+KjZAh</latexit><latexit sha1_base64="K3SAkvXn+lryYWGHf203kFgdJgw=">AAAB8XicbVDLSgNBEOz1GeMr6tHLYBA8hV0RzDGgB48RzAOTEGYnnWTI7Owy0yuEkL/w4kERr/6NN//GSbIHTSxoKKq66e4KEyUt+f63t7a+sbm1ndvJ7+7tHxwWjo7rNk6NwJqIVWyaIbeopMYaSVLYTAzyKFTYCEc3M7/xhMbKWD/QOMFOxAda9qXg5KTH9i0q4qzRDbqFol/y52CrJMhIETJUu4Wvdi8WaYSahOLWtgI/oc6EG5JC4TTfTi0mXIz4AFuOah6h7UzmF0/ZuVN6rB8bV5rYXP09MeGRteModJ0Rp6Fd9mbif14rpX65M5E6SQm1WCzqp4pRzGbvs540KEiNHeHCSHcrE0NuuCAXUt6FECy/vErql6XALwX3V8VKOYsjB6dwBhcQwDVU4A6qUAMBGp7hFd486714797HonXNy2ZO4A+8zx+KjZAh</latexit><latexit sha1_base64="K3SAkvXn+lryYWGHf203kFgdJgw=">AAAB8XicbVDLSgNBEOz1GeMr6tHLYBA8hV0RzDGgB48RzAOTEGYnnWTI7Owy0yuEkL/w4kERr/6NN//GSbIHTSxoKKq66e4KEyUt+f63t7a+sbm1ndvJ7+7tHxwWjo7rNk6NwJqIVWyaIbeopMYaSVLYTAzyKFTYCEc3M7/xhMbKWD/QOMFOxAda9qXg5KTH9i0q4qzRDbqFol/y52CrJMhIETJUu4Wvdi8WaYSahOLWtgI/oc6EG5JC4TTfTi0mXIz4AFuOah6h7UzmF0/ZuVN6rB8bV5rYXP09MeGRteModJ0Rp6Fd9mbif14rpX65M5E6SQm1WCzqp4pRzGbvs540KEiNHeHCSHcrE0NuuCAXUt6FECy/vErql6XALwX3V8VKOYsjB6dwBhcQwDVU4A6qUAMBGp7hFd486714797HonXNy2ZO4A+8zx+KjZAh</latexit>




Parameter Server



(2) Distributed training without a centralized server

Parameter Server

…

Flower Cat

…

Flower Cat

LeakrW



Flower Cat

LeakrW



Flower Cat

(b) Distributed training without a centralized server

Figure 1: The deep leakage scenarios in two categories of classical multi-node training. The little reddemon appears in the location where the deep leakage might happen. When performing centralizedtraining, the parameter server is capable to steal all training data from gradients received from workernodes. While training in a decentralized manner (e.g., ring all reduce [30]), any participant can bemalicious and steal the training data from its neighbors.

original ones (Fig. 5). When the optimization finishes, the private training data (both inputs andlabels) will be fully revealed.

Conventional “shallow” leakages (property inference [27, 34] and generative model [13] using classlabels) requires extra label information and can only generate similar synthetic images. Our “deep”leakage is an optimization process and does not depend on any generative models; therefore, DLGdoes not require any other extra prior about the training set, instead, it can infer the label from sharedgradients and the results produced by DLG (both images and texts) are the exact original trainingsamples instead of synthetic look-alike alternatives. We evaluate the effectiveness of our algorithm onboth vision (image classification) and language tasks (masked language model). On various datasetsand tasks, DLG fully recovers the training data in just a few gradient steps. Such a deep leakagefrom gradients is first discovered and we want to raise people’s awareness of rethinking the safety ofgradients.

The deep leakage puts a severe challenge to the multi-node machine learning system. The fundamentalgradient sharing scheme, as shown in our work, is not always reliable to protect the privacy of thetraining data. In centralized distributed training (Fig. 1a), the parameter server, which usually doesnot store any training data, is able to steal local training data of all participants. For decentralizeddistributed training (Fig. 1b), it becomes even worse since any participant can steal its neighbors’private training data. To prevent the deep leakage, we demonstrate three defense strategies: gradientperturbation, low precision, and gradient compression. For gradient perturbation, we find bothGaussian and Laplacian noise with a scale higher than 10−2 would be a good defense. While halfprecision fails to protect, gradient compression successfully defends the attack with the prunedgradient is more than 20%.

Our contributions include:

• We demonstrate that it is possible to obtain the private training data from the publicly sharedgradients. To our best knowledge, DLG is the first algorithm achieving it.

• DLG only requires the gradients and can reveal pixel-wise accurate images and token-wisematching texts. While conventional approaches usually need extra information to attack andonly produce partial properties or synthetic alternatives.

• To prevent potential leakage of important data, we analyze the attack difficulties in varioussettings and discuss several defense strategies against the attack.

2

2 Related Work

2.1 Distributed Training

Training large machine learning models (e.g., deep neural networks) is computationally intensive. Inorder to finish the training process in a reasonable time, many studies worked on distributed trainingto speedup. There are many works that aim to improve the scalability of distributed training, both atthe algorithm level [7, 11, 16, 23, 32] and at the framework level [1, 2, 6, 29, 33]. Most of them adaptsynchronous SGD as the backbone because the stable performance while scaling up.

In general, distributed training can be classified into two categories: with a parameter server (central-ized) [15, 19, 23] and without a parameter server (decentralized) [3, 30, 33]). In both schemes, eachnode first performs the computation to update its local weights, and then sends gradients to othernodes. For the centralized mode, the gradients first get aggregated and then delivered back to eachnode. For decentralized mode, gradients are exchanged between neighboring nodes.

In many application scenarios, the training data is privacy-sensitive. For example, a patient’s medicalcondition can not be shared across hospitals. To avoid the sensitive information being leaked,collaborative learning has recently emerged [17, 18, 26] where two or more participants can jointlytrain a model while the training dataset never leave each participants’ local server. Only the gradientsare shared across the network. This technique has been used to train models for medical treatmentsacross multiple hospitals [18], analyze patient survival situations from various countries [17] andbuild predictive keyboards to improve typing experience [4, 20, 26].

2.2 “Shallow” Leakage from Gradients

Previous works have made some explorations on how to infer the information of training data fromgradients. For some layers, the gradients already leak certain level of information. For example,the embedding layer in language tasks only produces gradients for words occurred in training data,which reveals what words have been used in other participant’s training set [27]. But such leakageis “shallow”: The leaked words is unordered and and it is hard to infer the original sentence due toambiguity. Another case is fully connected layers, where observations of gradient updates can beused to infer output feature values. However, this cannot extend to convolutional layers because thesize of the features is far larger than the size of weights.

Some recent works develop learning-based methods to infer properties of the batch. They show that abinary classifier trained on gradients is able to determine whether an exact data record (membershipinference [27, 34]) or a data record with certain properties (property inference [27]) is included in theother participant’s’ batch. Furthermore, they train GAN models [10] to synthesis images look similarto training data from the gradient [9, 13, 27], but the attack is limited and only works when all classmembers look alike (e.g., face recognition).

3 Method

We show that it’s possible to steal an image pixel-wise and steal a sentence token-wise from thegradients. We focus on the standard synchronous distributed training: At each step t, every node isamples a minibatch (xt,i,yt,i) from its own dataset to compute the gradients

∇Wt,i =∂`(F (xt,i,Wt),yt,i)

∂Wt(1)

The gradients are averaged across the N servers and then used to update the weights:

∇Wt =1

N

N∑j

∇Wt,j ; Wt+1 =Wt − η∇Wt (2)

Given gradients∇Wt,k received from other participant k, we aim to steal participant k’s training data(xt,k,yt,k). Note F () and Wt are shared by default for synchronized distributed optimization.

3

f(),W,�W<latexit sha1_base64="B96ZSK9vvSqPDAS/5unvlM98V5I=">AAAB+HicbVBNS8NAEJ3Ur1o/GvXoZbEIFUpJRLDHgh48VrBNoQ1ls920SzebsLsRaugv8eJBEa/+FG/+G7dtDtr6YODx3gwz84KEM6Ud59sqbGxube8Ud0t7+weHZfvouKPiVBLaJjGPZTfAinImaFszzWk3kRRHAadeMLmZ+94jlYrF4kFPE+pHeCRYyAjWRhrY5bB6UfNq/VvKNUYeGtgVp+4sgNaJm5MK5GgN7K/+MCZpRIUmHCvVc51E+xmWmhFOZ6V+qmiCyQSPaM9QgSOq/Gxx+AydG2WIwliaEhot1N8TGY6UmkaB6YywHqtVby7+5/VSHTb8jIkk1VSQ5aIw5UjHaJ4CGjJJieZTQzCRzNyKyBhLTLTJqmRCcFdfXiedy7rr1N37q0qzkcdRhFM4gyq4cA1NuIMWtIFACs/wCm/Wk/VivVsfy9aClc+cwB9Ynz89a5F6</latexit><latexit sha1_base64="B96ZSK9vvSqPDAS/5unvlM98V5I=">AAAB+HicbVBNS8NAEJ3Ur1o/GvXoZbEIFUpJRLDHgh48VrBNoQ1ls920SzebsLsRaugv8eJBEa/+FG/+G7dtDtr6YODx3gwz84KEM6Ud59sqbGxube8Ud0t7+weHZfvouKPiVBLaJjGPZTfAinImaFszzWk3kRRHAadeMLmZ+94jlYrF4kFPE+pHeCRYyAjWRhrY5bB6UfNq/VvKNUYeGtgVp+4sgNaJm5MK5GgN7K/+MCZpRIUmHCvVc51E+xmWmhFOZ6V+qmiCyQSPaM9QgSOq/Gxx+AydG2WIwliaEhot1N8TGY6UmkaB6YywHqtVby7+5/VSHTb8jIkk1VSQ5aIw5UjHaJ4CGjJJieZTQzCRzNyKyBhLTLTJqmRCcFdfXiedy7rr1N37q0qzkcdRhFM4gyq4cA1NuIMWtIFACs/wCm/Wk/VivVsfy9aClc+cwB9Ynz89a5F6</latexit><latexit sha1_base64="B96ZSK9vvSqPDAS/5unvlM98V5I=">AAAB+HicbVBNS8NAEJ3Ur1o/GvXoZbEIFUpJRLDHgh48VrBNoQ1ls920SzebsLsRaugv8eJBEa/+FG/+G7dtDtr6YODx3gwz84KEM6Ud59sqbGxube8Ud0t7+weHZfvouKPiVBLaJjGPZTfAinImaFszzWk3kRRHAadeMLmZ+94jlYrF4kFPE+pHeCRYyAjWRhrY5bB6UfNq/VvKNUYeGtgVp+4sgNaJm5MK5GgN7K/+MCZpRIUmHCvVc51E+xmWmhFOZ6V+qmiCyQSPaM9QgSOq/Gxx+AydG2WIwliaEhot1N8TGY6UmkaB6YywHqtVby7+5/VSHTb8jIkk1VSQ5aIw5UjHaJ4CGjJJieZTQzCRzNyKyBhLTLTJqmRCcFdfXiedy7rr1N37q0qzkcdRhFM4gyq4cA1NuIMWtIFACs/wCm/Wk/VivVsfy9aClc+cwB9Ynz89a5F6</latexit><latexit sha1_base64="hP+6LrUf2d3tZaldqaQQvEKMXyw=">AAAB2XicbZDNSgMxFIXv1L86Vq1rN8EiuCozbnQpuHFZwbZCO5RM5k4bmskMyR2hDH0BF25EfC93vo3pz0JbDwQ+zknIvSculLQUBN9ebWd3b/+gfugfNfzjk9Nmo2fz0gjsilzl5jnmFpXU2CVJCp8LgzyLFfbj6f0i77+gsTLXTzQrMMr4WMtUCk7O6oyaraAdLMW2IVxDC9YaNb+GSS7KDDUJxa0dhEFBUcUNSaFw7g9LiwUXUz7GgUPNM7RRtRxzzi6dk7A0N+5oYkv394uKZ9bOstjdzDhN7Ga2MP/LBiWlt1EldVESarH6KC0Vo5wtdmaJNChIzRxwYaSblYkJN1yQa8Z3HYSbG29D77odBu3wMYA6nMMFXEEIN3AHD9CBLghI4BXevYn35n2suqp569LO4I+8zx84xIo4</latexit><latexit sha1_base64="GdWVN1k7EtO+FhtRAVNVXIsyRRw=">AAAB7XicbZDNSgMxFIXv1L9aqx3dugkWoUIpM250KejCZQXbKbRDyaR32tBMZkgyQi19EjcuFPF13Pk2pj8LbT0Q+Dgn4d6cKBNcG8/7dgpb2zu7e8X90kH58KjiHpfbOs0VwxZLRao6EdUouMSW4UZgJ1NIk0hgEI1v53nwhErzVD6aSYZhQoeSx5xRY62+W4lrF/Wg3rtDYSgJSN+teg1vIbIJ/gqqsFKz7371BinLE5SGCap11/cyE06pMpwJnJV6ucaMsjEdYteipAnqcLpYfEbOrTMgcarskYYs3N8vpjTRepJE9mZCzUivZ3Pzv6ybm/g6nHKZ5QYlWw6Kc0FMSuYtkAFXyIyYWKBMcbsrYSOqKDO2q5ItwV//8ia0Lxu+1/AfPCjCKZxBDXy4ghu4hya0gEEOL/AG786z8+p8LOsqOKveTuCPnM8f/MCQIQ==</latexit><latexit sha1_base64="GdWVN1k7EtO+FhtRAVNVXIsyRRw=">AAAB7XicbZDNSgMxFIXv1L9aqx3dugkWoUIpM250KejCZQXbKbRDyaR32tBMZkgyQi19EjcuFPF13Pk2pj8LbT0Q+Dgn4d6cKBNcG8/7dgpb2zu7e8X90kH58KjiHpfbOs0VwxZLRao6EdUouMSW4UZgJ1NIk0hgEI1v53nwhErzVD6aSYZhQoeSx5xRY62+W4lrF/Wg3rtDYSgJSN+teg1vIbIJ/gqqsFKz7371BinLE5SGCap11/cyE06pMpwJnJV6ucaMsjEdYteipAnqcLpYfEbOrTMgcarskYYs3N8vpjTRepJE9mZCzUivZ3Pzv6ybm/g6nHKZ5QYlWw6Kc0FMSuYtkAFXyIyYWKBMcbsrYSOqKDO2q5ItwV//8ia0Lxu+1/AfPCjCKZxBDXy4ghu4hya0gEEOL/AG786z8+p8LOsqOKveTuCPnM8f/MCQIQ==</latexit><latexit sha1_base64="3KYNHBQTHEpbUyiCrWc9JnS/jLY=">AAAB+HicbVDLSgNBEOyNrxgfWfXoZTAIEULY9WKOAT14jGCyQrKE2clsMmT2wUyvEEO+xIsHRbz6Kd78GyfJHjSxoKGo6qa7K0il0Og431ZhY3Nre6e4W9rbPzgs20fHHZ1kivE2S2SiHgKquRQxb6NAyR9SxWkUSO4F4+u57z1ypUUS3+Mk5X5Eh7EIBaNopL5dDqsXNa/Wu+ESKfFI3644dWcBsk7cnFQgR6tvf/UGCcsiHiOTVOuu66ToT6lCwSSflXqZ5illYzrkXUNjGnHtTxeHz8i5UQYkTJSpGMlC/T0xpZHWkygwnRHFkV715uJ/XjfDsOFPRZxmyGO2XBRmkmBC5imQgVCcoZwYQpkS5lbCRlRRhiarkgnBXX15nXQu665Td++cSrORx1GEUziDKrhwBU24hRa0gUEGz/AKb9aT9WK9Wx/L1oKVz5zAH1ifPzwrkXY=</latexit><latexit sha1_base64="B96ZSK9vvSqPDAS/5unvlM98V5I=">AAAB+HicbVBNS8NAEJ3Ur1o/GvXoZbEIFUpJRLDHgh48VrBNoQ1ls920SzebsLsRaugv8eJBEa/+FG/+G7dtDtr6YODx3gwz84KEM6Ud59sqbGxube8Ud0t7+weHZfvouKPiVBLaJjGPZTfAinImaFszzWk3kRRHAadeMLmZ+94jlYrF4kFPE+pHeCRYyAjWRhrY5bB6UfNq/VvKNUYeGtgVp+4sgNaJm5MK5GgN7K/+MCZpRIUmHCvVc51E+xmWmhFOZ6V+qmiCyQSPaM9QgSOq/Gxx+AydG2WIwliaEhot1N8TGY6UmkaB6YywHqtVby7+5/VSHTb8jIkk1VSQ5aIw5UjHaJ4CGjJJieZTQzCRzNyKyBhLTLTJqmRCcFdfXiedy7rr1N37q0qzkcdRhFM4gyq4cA1NuIMWtIFACs/wCm/Wk/VivVsfy9aClc+cwB9Ynz89a5F6</latexit><latexit sha1_base64="B96ZSK9vvSqPDAS/5unvlM98V5I=">AAAB+HicbVBNS8NAEJ3Ur1o/GvXoZbEIFUpJRLDHgh48VrBNoQ1ls920SzebsLsRaugv8eJBEa/+FG/+G7dtDtr6YODx3gwz84KEM6Ud59sqbGxube8Ud0t7+weHZfvouKPiVBLaJjGPZTfAinImaFszzWk3kRRHAadeMLmZ+94jlYrF4kFPE+pHeCRYyAjWRhrY5bB6UfNq/VvKNUYeGtgVp+4sgNaJm5MK5GgN7K/+MCZpRIUmHCvVc51E+xmWmhFOZ6V+qmiCyQSPaM9QgSOq/Gxx+AydG2WIwliaEhot1N8TGY6UmkaB6YywHqtVby7+5/VSHTb8jIkk1VSQ5aIw5UjHaJ4CGjJJieZTQzCRzNyKyBhLTLTJqmRCcFdfXiedy7rr1N37q0qzkcdRhFM4gyq4cA1NuIMWtIFACs/wCm/Wk/VivVsfy9aClc+cwB9Ynz89a5F6</latexit><latexit sha1_base64="B96ZSK9vvSqPDAS/5unvlM98V5I=">AAAB+HicbVBNS8NAEJ3Ur1o/GvXoZbEIFUpJRLDHgh48VrBNoQ1ls920SzebsLsRaugv8eJBEa/+FG/+G7dtDtr6YODx3gwz84KEM6Ud59sqbGxube8Ud0t7+weHZfvouKPiVBLaJjGPZTfAinImaFszzWk3kRRHAadeMLmZ+94jlYrF4kFPE+pHeCRYyAjWRhrY5bB6UfNq/VvKNUYeGtgVp+4sgNaJm5MK5GgN7K/+MCZpRIUmHCvVc51E+xmWmhFOZ6V+qmiCyQSPaM9QgSOq/Gxx+AydG2WIwliaEhot1N8TGY6UmkaB6YywHqtVby7+5/VSHTb8jIkk1VSQ5aIw5UjHaJ4CGjJJieZTQzCRzNyKyBhLTLTJqmRCcFdfXiedy7rr1N37q0qzkcdRhFM4gyq4cA1NuIMWtIFACs/wCm/Wk/VivVsfy9aClc+cwB9Ynz89a5F6</latexit><latexit sha1_base64="B96ZSK9vvSqPDAS/5unvlM98V5I=">AAAB+HicbVBNS8NAEJ3Ur1o/GvXoZbEIFUpJRLDHgh48VrBNoQ1ls920SzebsLsRaugv8eJBEa/+FG/+G7dtDtr6YODx3gwz84KEM6Ud59sqbGxube8Ud0t7+weHZfvouKPiVBLaJjGPZTfAinImaFszzWk3kRRHAadeMLmZ+94jlYrF4kFPE+pHeCRYyAjWRhrY5bB6UfNq/VvKNUYeGtgVp+4sgNaJm5MK5GgN7K/+MCZpRIUmHCvVc51E+xmWmhFOZ6V+qmiCyQSPaM9QgSOq/Gxx+AydG2WIwliaEhot1N8TGY6UmkaB6YywHqtVby7+5/VSHTb8jIkk1VSQ5aIw5UjHaJ4CGjJJieZTQzCRzNyKyBhLTLTJqmRCcFdfXiedy7rr1N37q0qzkcdRhFM4gyq4cA1NuIMWtIFACs/wCm/Wk/VivVsfy9aClc+cwB9Ynz89a5F6</latexit><latexit sha1_base64="B96ZSK9vvSqPDAS/5unvlM98V5I=">AAAB+HicbVBNS8NAEJ3Ur1o/GvXoZbEIFUpJRLDHgh48VrBNoQ1ls920SzebsLsRaugv8eJBEa/+FG/+G7dtDtr6YODx3gwz84KEM6Ud59sqbGxube8Ud0t7+weHZfvouKPiVBLaJjGPZTfAinImaFszzWk3kRRHAadeMLmZ+94jlYrF4kFPE+pHeCRYyAjWRhrY5bB6UfNq/VvKNUYeGtgVp+4sgNaJm5MK5GgN7K/+MCZpRIUmHCvVc51E+xmWmhFOZ6V+qmiCyQSPaM9QgSOq/Gxx+AydG2WIwliaEhot1N8TGY6UmkaB6YywHqtVby7+5/VSHTb8jIkk1VSQ5aIw5UjHaJ4CGjJJieZTQzCRzNyKyBhLTLTJqmRCcFdfXiedy7rr1N37q0qzkcdRhFM4gyq4cA1NuIMWtIFACs/wCm/Wk/VivVsfy9aClc+cwB9Ynz89a5F6</latexit><latexit sha1_base64="hP+6LrUf2d3tZaldqaQQvEKMXyw=">AAAB2XicbZDNSgMxFIXv1L86Vq1rN8EiuCozbnQpuHFZwbZCO5RM5k4bmskMyR2hDH0BF25EfC93vo3pz0JbDwQ+zknIvSculLQUBN9ebWd3b/+gfugfNfzjk9Nmo2fz0gjsilzl5jnmFpXU2CVJCp8LgzyLFfbj6f0i77+gsTLXTzQrMMr4WMtUCk7O6oyaraAdLMW2IVxDC9YaNb+GSS7KDDUJxa0dhEFBUcUNSaFw7g9LiwUXUz7GgUPNM7RRtRxzzi6dk7A0N+5oYkv394uKZ9bOstjdzDhN7Ga2MP/LBiWlt1EldVESarH6KC0Vo5wtdmaJNChIzRxwYaSblYkJN1yQa8Z3HYSbG29D77odBu3wMYA6nMMFXEEIN3AHD9CBLghI4BXevYn35n2suqp569LO4I+8zx84xIo4</latexit><latexit sha1_base64="GdWVN1k7EtO+FhtRAVNVXIsyRRw=">AAAB7XicbZDNSgMxFIXv1L9aqx3dugkWoUIpM250KejCZQXbKbRDyaR32tBMZkgyQi19EjcuFPF13Pk2pj8LbT0Q+Dgn4d6cKBNcG8/7dgpb2zu7e8X90kH58KjiHpfbOs0VwxZLRao6EdUouMSW4UZgJ1NIk0hgEI1v53nwhErzVD6aSYZhQoeSx5xRY62+W4lrF/Wg3rtDYSgJSN+teg1vIbIJ/gqqsFKz7371BinLE5SGCap11/cyE06pMpwJnJV6ucaMsjEdYteipAnqcLpYfEbOrTMgcarskYYs3N8vpjTRepJE9mZCzUivZ3Pzv6ybm/g6nHKZ5QYlWw6Kc0FMSuYtkAFXyIyYWKBMcbsrYSOqKDO2q5ItwV//8ia0Lxu+1/AfPCjCKZxBDXy4ghu4hya0gEEOL/AG786z8+p8LOsqOKveTuCPnM8f/MCQIQ==</latexit><latexit sha1_base64="GdWVN1k7EtO+FhtRAVNVXIsyRRw=">AAAB7XicbZDNSgMxFIXv1L9aqx3dugkWoUIpM250KejCZQXbKbRDyaR32tBMZkgyQi19EjcuFPF13Pk2pj8LbT0Q+Dgn4d6cKBNcG8/7dgpb2zu7e8X90kH58KjiHpfbOs0VwxZLRao6EdUouMSW4UZgJ1NIk0hgEI1v53nwhErzVD6aSYZhQoeSx5xRY62+W4lrF/Wg3rtDYSgJSN+teg1vIbIJ/gqqsFKz7371BinLE5SGCap11/cyE06pMpwJnJV6ucaMsjEdYteipAnqcLpYfEbOrTMgcarskYYs3N8vpjTRepJE9mZCzUivZ3Pzv6ybm/g6nHKZ5QYlWw6Kc0FMSuYtkAFXyIyYWKBMcbsrYSOqKDO2q5ItwV//8ia0Lxu+1/AfPCjCKZxBDXy4ghu4hya0gEEOL/AG786z8+p8LOsqOKveTuCPnM8f/MCQIQ==</latexit><latexit sha1_base64="3KYNHBQTHEpbUyiCrWc9JnS/jLY=">AAAB+HicbVDLSgNBEOyNrxgfWfXoZTAIEULY9WKOAT14jGCyQrKE2clsMmT2wUyvEEO+xIsHRbz6Kd78GyfJHjSxoKGo6qa7K0il0Og431ZhY3Nre6e4W9rbPzgs20fHHZ1kivE2S2SiHgKquRQxb6NAyR9SxWkUSO4F4+u57z1ypUUS3+Mk5X5Eh7EIBaNopL5dDqsXNa/Wu+ESKfFI3644dWcBsk7cnFQgR6tvf/UGCcsiHiOTVOuu66ToT6lCwSSflXqZ5illYzrkXUNjGnHtTxeHz8i5UQYkTJSpGMlC/T0xpZHWkygwnRHFkV715uJ/XjfDsOFPRZxmyGO2XBRmkmBC5imQgVCcoZwYQpkS5lbCRlRRhiarkgnBXX15nXQu665Td++cSrORx1GEUziDKrhwBU24hRa0gUEGz/AKb9aT9WK9Wx/L1oKVz5zAH1ifPzwrkXY=</latexit><latexit sha1_base64="B96ZSK9vvSqPDAS/5unvlM98V5I=">AAAB+HicbVBNS8NAEJ3Ur1o/GvXoZbEIFUpJRLDHgh48VrBNoQ1ls920SzebsLsRaugv8eJBEa/+FG/+G7dtDtr6YODx3gwz84KEM6Ud59sqbGxube8Ud0t7+weHZfvouKPiVBLaJjGPZTfAinImaFszzWk3kRRHAadeMLmZ+94jlYrF4kFPE+pHeCRYyAjWRhrY5bB6UfNq/VvKNUYeGtgVp+4sgNaJm5MK5GgN7K/+MCZpRIUmHCvVc51E+xmWmhFOZ6V+qmiCyQSPaM9QgSOq/Gxx+AydG2WIwliaEhot1N8TGY6UmkaB6YywHqtVby7+5/VSHTb8jIkk1VSQ5aIw5UjHaJ4CGjJJieZTQzCRzNyKyBhLTLTJqmRCcFdfXiedy7rr1N37q0qzkcdRhFM4gyq4cA1NuIMWtIFACs/wCm/Wk/VivVsfy9aClc+cwB9Ynz89a5F6</latexit><latexit sha1_base64="B96ZSK9vvSqPDAS/5unvlM98V5I=">AAAB+HicbVBNS8NAEJ3Ur1o/GvXoZbEIFUpJRLDHgh48VrBNoQ1ls920SzebsLsRaugv8eJBEa/+FG/+G7dtDtr6YODx3gwz84KEM6Ud59sqbGxube8Ud0t7+weHZfvouKPiVBLaJjGPZTfAinImaFszzWk3kRRHAadeMLmZ+94jlYrF4kFPE+pHeCRYyAjWRhrY5bB6UfNq/VvKNUYeGtgVp+4sgNaJm5MK5GgN7K/+MCZpRIUmHCvVc51E+xmWmhFOZ6V+qmiCyQSPaM9QgSOq/Gxx+AydG2WIwliaEhot1N8TGY6UmkaB6YywHqtVby7+5/VSHTb8jIkk1VSQ5aIw5UjHaJ4CGjJJieZTQzCRzNyKyBhLTLTJqmRCcFdfXiedy7rr1N37q0qzkcdRhFM4gyq4cA1NuIMWtIFACs/wCm/Wk/VivVsfy9aClc+cwB9Ynz89a5F6</latexit><latexit sha1_base64="B96ZSK9vvSqPDAS/5unvlM98V5I=">AAAB+HicbVBNS8NAEJ3Ur1o/GvXoZbEIFUpJRLDHgh48VrBNoQ1ls920SzebsLsRaugv8eJBEa/+FG/+G7dtDtr6YODx3gwz84KEM6Ud59sqbGxube8Ud0t7+weHZfvouKPiVBLaJjGPZTfAinImaFszzWk3kRRHAadeMLmZ+94jlYrF4kFPE+pHeCRYyAjWRhrY5bB6UfNq/VvKNUYeGtgVp+4sgNaJm5MK5GgN7K/+MCZpRIUmHCvVc51E+xmWmhFOZ6V+qmiCyQSPaM9QgSOq/Gxx+AydG2WIwliaEhot1N8TGY6UmkaB6YywHqtVby7+5/VSHTb8jIkk1VSQ5aIw5UjHaJ4CGjJJieZTQzCRzNyKyBhLTLTJqmRCcFdfXiedy7rr1N37q0qzkcdRhFM4gyq4cA1NuIMWtIFACs/wCm/Wk/VivVsfy9aClc+cwB9Ynz89a5F6</latexit><latexit sha1_base64="B96ZSK9vvSqPDAS/5unvlM98V5I=">AAAB+HicbVBNS8NAEJ3Ur1o/GvXoZbEIFUpJRLDHgh48VrBNoQ1ls920SzebsLsRaugv8eJBEa/+FG/+G7dtDtr6YODx3gwz84KEM6Ud59sqbGxube8Ud0t7+weHZfvouKPiVBLaJjGPZTfAinImaFszzWk3kRRHAadeMLmZ+94jlYrF4kFPE+pHeCRYyAjWRhrY5bB6UfNq/VvKNUYeGtgVp+4sgNaJm5MK5GgN7K/+MCZpRIUmHCvVc51E+xmWmhFOZ6V+qmiCyQSPaM9QgSOq/Gxx+AydG2WIwliaEhot1N8TGY6UmkaB6YywHqtVby7+5/VSHTb8jIkk1VSQ5aIw5UjHaJ4CGjJJieZTQzCRzNyKyBhLTLTJqmRCcFdfXiedy7rr1N37q0qzkcdRhFM4gyq4cA1NuIMWtIFACs/wCm/Wk/VivVsfy9aClc+cwB9Ynz89a5F6</latexit><latexit sha1_base64="B96ZSK9vvSqPDAS/5unvlM98V5I=">AAAB+HicbVBNS8NAEJ3Ur1o/GvXoZbEIFUpJRLDHgh48VrBNoQ1ls920SzebsLsRaugv8eJBEa/+FG/+G7dtDtr6YODx3gwz84KEM6Ud59sqbGxube8Ud0t7+weHZfvouKPiVBLaJjGPZTfAinImaFszzWk3kRRHAadeMLmZ+94jlYrF4kFPE+pHeCRYyAjWRhrY5bB6UfNq/VvKNUYeGtgVp+4sgNaJm5MK5GgN7K/+MCZpRIUmHCvVc51E+xmWmhFOZ6V+qmiCyQSPaM9QgSOq/Gxx+AydG2WIwliaEhot1N8TGY6UmkaB6YywHqtVby7+5/VSHTb8jIkk1VSQ5aIw5UjHaJ4CGjJJieZTQzCRzNyKyBhLTLTJqmRCcFdfXiedy7rr1N37q0qzkcdRhFM4gyq4cA1NuIMWtIFACs/wCm/Wk/VivVsfy9aClc+cwB9Ynz89a5F6</latexit><latexit sha1_base64="B96ZSK9vvSqPDAS/5unvlM98V5I=">AAAB+HicbVBNS8NAEJ3Ur1o/GvXoZbEIFUpJRLDHgh48VrBNoQ1ls920SzebsLsRaugv8eJBEa/+FG/+G7dtDtr6YODx3gwz84KEM6Ud59sqbGxube8Ud0t7+weHZfvouKPiVBLaJjGPZTfAinImaFszzWk3kRRHAadeMLmZ+94jlYrF4kFPE+pHeCRYyAjWRhrY5bB6UfNq/VvKNUYeGtgVp+4sgNaJm5MK5GgN7K/+MCZpRIUmHCvVc51E+xmWmhFOZ6V+qmiCyQSPaM9QgSOq/Gxx+AydG2WIwliaEhot1N8TGY6UmkaB6YywHqtVby7+5/VSHTb8jIkk1VSQ5aIw5UjHaJ4CGjJJieZTQzCRzNyKyBhLTLTJqmRCcFdfXiedy7rr1N37q0qzkcdRhFM4gyq4cA1NuIMWtIFACs/wCm/Wk/VivVsfy9aClc+cwB9Ynz89a5F6</latexit>

GT: Cat

Normal Client

Malicious Attacker

Differentiable Model f(x; W) Pred Loss

Shared Information:

minimize||�W 0 ��W ||22<latexit sha1_base64="JFN8C39v9ltXBBEGnSa3ZlBpMJE=">AAACFnicbVDLSgMxFM34rPU16tJNsIhuWmaKYJcFXbisYB/Q1pJJb9vQJDMkGaFO+xVu/BU3LhRxK+78G9OHoK0HAodz7uXmnCDiTBvP+3KWlldW19ZTG+nNre2dXXdvv6LDWFEo05CHqhYQDZxJKBtmONQiBUQEHKpB/2LsV+9AaRbKGzOIoClIV7IOo8RYqeVmG4KYnhKJYJIJdg+j4bBxCdwQXD3BWfzDh8NW/jbfcjNezpsALxJ/RjJohlLL/Wy0QxoLkIZyonXd9yLTTIgyjHIYpRuxhojQPulC3VJJBOhmMok1wsdWaeNOqOyTBk/U3xsJEVoPRGAnxyH0vDcW//PqsekUmgmTUWxA0umhTsyxCfG4I9xmCqjhA0sIVcz+FdMeUYQa22TaluDPR14klXzO93L+9VmmWJjVkUKH6AidIh+doyK6QiVURhQ9oCf0gl6dR+fZeXPep6NLzmznAP2B8/ENWgyewQ==</latexit><latexit sha1_base64="JFN8C39v9ltXBBEGnSa3ZlBpMJE=">AAACFnicbVDLSgMxFM34rPU16tJNsIhuWmaKYJcFXbisYB/Q1pJJb9vQJDMkGaFO+xVu/BU3LhRxK+78G9OHoK0HAodz7uXmnCDiTBvP+3KWlldW19ZTG+nNre2dXXdvv6LDWFEo05CHqhYQDZxJKBtmONQiBUQEHKpB/2LsV+9AaRbKGzOIoClIV7IOo8RYqeVmG4KYnhKJYJIJdg+j4bBxCdwQXD3BWfzDh8NW/jbfcjNezpsALxJ/RjJohlLL/Wy0QxoLkIZyonXd9yLTTIgyjHIYpRuxhojQPulC3VJJBOhmMok1wsdWaeNOqOyTBk/U3xsJEVoPRGAnxyH0vDcW//PqsekUmgmTUWxA0umhTsyxCfG4I9xmCqjhA0sIVcz+FdMeUYQa22TaluDPR14klXzO93L+9VmmWJjVkUKH6AidIh+doyK6QiVURhQ9oCf0gl6dR+fZeXPep6NLzmznAP2B8/ENWgyewQ==</latexit><latexit sha1_base64="JFN8C39v9ltXBBEGnSa3ZlBpMJE=">AAACFnicbVDLSgMxFM34rPU16tJNsIhuWmaKYJcFXbisYB/Q1pJJb9vQJDMkGaFO+xVu/BU3LhRxK+78G9OHoK0HAodz7uXmnCDiTBvP+3KWlldW19ZTG+nNre2dXXdvv6LDWFEo05CHqhYQDZxJKBtmONQiBUQEHKpB/2LsV+9AaRbKGzOIoClIV7IOo8RYqeVmG4KYnhKJYJIJdg+j4bBxCdwQXD3BWfzDh8NW/jbfcjNezpsALxJ/RjJohlLL/Wy0QxoLkIZyonXd9yLTTIgyjHIYpRuxhojQPulC3VJJBOhmMok1wsdWaeNOqOyTBk/U3xsJEVoPRGAnxyH0vDcW//PqsekUmgmTUWxA0umhTsyxCfG4I9xmCqjhA0sIVcz+FdMeUYQa22TaluDPR14klXzO93L+9VmmWJjVkUKH6AidIh+doyK6QiVURhQ9oCf0gl6dR+fZeXPep6NLzmznAP2B8/ENWgyewQ==</latexit><latexit sha1_base64="JFN8C39v9ltXBBEGnSa3ZlBpMJE=">AAACFnicbVDLSgMxFM34rPU16tJNsIhuWmaKYJcFXbisYB/Q1pJJb9vQJDMkGaFO+xVu/BU3LhRxK+78G9OHoK0HAodz7uXmnCDiTBvP+3KWlldW19ZTG+nNre2dXXdvv6LDWFEo05CHqhYQDZxJKBtmONQiBUQEHKpB/2LsV+9AaRbKGzOIoClIV7IOo8RYqeVmG4KYnhKJYJIJdg+j4bBxCdwQXD3BWfzDh8NW/jbfcjNezpsALxJ/RjJohlLL/Wy0QxoLkIZyonXd9yLTTIgyjHIYpRuxhojQPulC3VJJBOhmMok1wsdWaeNOqOyTBk/U3xsJEVoPRGAnxyH0vDcW//PqsekUmgmTUWxA0umhTsyxCfG4I9xmCqjhA0sIVcz+FdMeUYQa22TaluDPR14klXzO93L+9VmmWJjVkUKH6AidIh+doyK6QiVURhQ9oCf0gl6dR+fZeXPep6NLzmznAP2B8/ENWgyewQ==</latexit>

Dum

my

Labe

l

Differentiable Model f(x; W) Pred Loss

�W 0<latexit sha1_base64="D+xByVWKvot2StJCmToE4fD2eYY=">AAAB8HicbVBNS8NAEJ3Ur1q/qh69LBbRU0lEsMeCHjxWsB/ShrLZbtqlu0nYnQgl9Fd48aCIV3+ON/+N2zYHbX0w8Hhvhpl5QSKFQdf9dgpr6xubW8Xt0s7u3v5B+fCoZeJUM95ksYx1J6CGSxHxJgqUvJNoTlUgeTsY38z89hPXRsTRA04S7is6jEQoGEUrPfZuuURK2uf9csWtunOQVeLlpAI5Gv3yV28Qs1TxCJmkxnQ9N0E/oxoFk3xa6qWGJ5SN6ZB3LY2o4sbP5gdPyZlVBiSMta0IyVz9PZFRZcxEBbZTURyZZW8m/ud1UwxrfiaiJEUescWiMJUEYzL7ngyE5gzlxBLKtLC3EjaimjK0GZVsCN7yy6ukdVn13Kp3f1Wp1/I4inACp3ABHlxDHe6gAU1goOAZXuHN0c6L8+58LFoLTj5zDH/gfP4Aw0qPrg==</latexit><latexit sha1_base64="D+xByVWKvot2StJCmToE4fD2eYY=">AAAB8HicbVBNS8NAEJ3Ur1q/qh69LBbRU0lEsMeCHjxWsB/ShrLZbtqlu0nYnQgl9Fd48aCIV3+ON/+N2zYHbX0w8Hhvhpl5QSKFQdf9dgpr6xubW8Xt0s7u3v5B+fCoZeJUM95ksYx1J6CGSxHxJgqUvJNoTlUgeTsY38z89hPXRsTRA04S7is6jEQoGEUrPfZuuURK2uf9csWtunOQVeLlpAI5Gv3yV28Qs1TxCJmkxnQ9N0E/oxoFk3xa6qWGJ5SN6ZB3LY2o4sbP5gdPyZlVBiSMta0IyVz9PZFRZcxEBbZTURyZZW8m/ud1UwxrfiaiJEUescWiMJUEYzL7ngyE5gzlxBLKtLC3EjaimjK0GZVsCN7yy6ukdVn13Kp3f1Wp1/I4inACp3ABHlxDHe6gAU1goOAZXuHN0c6L8+58LFoLTj5zDH/gfP4Aw0qPrg==</latexit><latexit sha1_base64="D+xByVWKvot2StJCmToE4fD2eYY=">AAAB8HicbVBNS8NAEJ3Ur1q/qh69LBbRU0lEsMeCHjxWsB/ShrLZbtqlu0nYnQgl9Fd48aCIV3+ON/+N2zYHbX0w8Hhvhpl5QSKFQdf9dgpr6xubW8Xt0s7u3v5B+fCoZeJUM95ksYx1J6CGSxHxJgqUvJNoTlUgeTsY38z89hPXRsTRA04S7is6jEQoGEUrPfZuuURK2uf9csWtunOQVeLlpAI5Gv3yV28Qs1TxCJmkxnQ9N0E/oxoFk3xa6qWGJ5SN6ZB3LY2o4sbP5gdPyZlVBiSMta0IyVz9PZFRZcxEBbZTURyZZW8m/ud1UwxrfiaiJEUescWiMJUEYzL7ngyE5gzlxBLKtLC3EjaimjK0GZVsCN7yy6ukdVn13Kp3f1Wp1/I4inACp3ABHlxDHe6gAU1goOAZXuHN0c6L8+58LFoLTj5zDH/gfP4Aw0qPrg==</latexit><latexit sha1_base64="D+xByVWKvot2StJCmToE4fD2eYY=">AAAB8HicbVBNS8NAEJ3Ur1q/qh69LBbRU0lEsMeCHjxWsB/ShrLZbtqlu0nYnQgl9Fd48aCIV3+ON/+N2zYHbX0w8Hhvhpl5QSKFQdf9dgpr6xubW8Xt0s7u3v5B+fCoZeJUM95ksYx1J6CGSxHxJgqUvJNoTlUgeTsY38z89hPXRsTRA04S7is6jEQoGEUrPfZuuURK2uf9csWtunOQVeLlpAI5Gv3yV28Qs1TxCJmkxnQ9N0E/oxoFk3xa6qWGJ5SN6ZB3LY2o4sbP5gdPyZlVBiSMta0IyVz9PZFRZcxEBbZTURyZZW8m/ud1UwxrfiaiJEUescWiMJUEYzL7ngyE5gzlxBLKtLC3EjaimjK0GZVsCN7yy6ukdVn13Kp3f1Wp1/I4inACp3ABHlxDHe6gAU1goOAZXuHN0c6L8+58LFoLTj5zDH/gfP4Aw0qPrg==</latexit>

[0.17, 0.2, 0.73]Differentiable Model f(x, W) Pred’ Loss’

rWLoss<latexit sha1_base64="bhOtvWaf4NML8EAFWbV91QG2/q8=">AAAB+HicbVBNS8NAEJ3Ur1o/GvXoZbEInkoigj0WvHjwUMF+QBvCZrtpl252w+5GqKG/xIsHRbz6U7z5b9y2OWjrg4HHezPMzItSzrTxvG+ntLG5tb1T3q3s7R8cVt2j446WmSK0TSSXqhdhTTkTtG2Y4bSXKoqTiNNuNLmZ+91HqjST4sFMUxokeCRYzAg2Vgrd6kDgiOMw787QndQ6dGte3VsArRO/IDUo0Ardr8FQkiyhwhCOte77XmqCHCvDCKezyiDTNMVkgke0b6nACdVBvjh8hs6tMkSxVLaEQQv190SOE62nSWQ7E2zGetWbi/95/czEjSBnIs0MFWS5KM44MhLNU0BDpigxfGoJJorZWxEZY4WJsVlVbAj+6svrpHNZ9726f39VazaKOMpwCmdwAT5cQxNuoQVtIJDBM7zCm/PkvDjvzseyteQUMyfwB87nD5gJkwA=</latexit><latexit sha1_base64="bhOtvWaf4NML8EAFWbV91QG2/q8=">AAAB+HicbVBNS8NAEJ3Ur1o/GvXoZbEInkoigj0WvHjwUMF+QBvCZrtpl252w+5GqKG/xIsHRbz6U7z5b9y2OWjrg4HHezPMzItSzrTxvG+ntLG5tb1T3q3s7R8cVt2j446WmSK0TSSXqhdhTTkTtG2Y4bSXKoqTiNNuNLmZ+91HqjST4sFMUxokeCRYzAg2Vgrd6kDgiOMw787QndQ6dGte3VsArRO/IDUo0Ardr8FQkiyhwhCOte77XmqCHCvDCKezyiDTNMVkgke0b6nACdVBvjh8hs6tMkSxVLaEQQv190SOE62nSWQ7E2zGetWbi/95/czEjSBnIs0MFWS5KM44MhLNU0BDpigxfGoJJorZWxEZY4WJsVlVbAj+6svrpHNZ9726f39VazaKOMpwCmdwAT5cQxNuoQVtIJDBM7zCm/PkvDjvzseyteQUMyfwB87nD5gJkwA=</latexit><latexit sha1_base64="bhOtvWaf4NML8EAFWbV91QG2/q8=">AAAB+HicbVBNS8NAEJ3Ur1o/GvXoZbEInkoigj0WvHjwUMF+QBvCZrtpl252w+5GqKG/xIsHRbz6U7z5b9y2OWjrg4HHezPMzItSzrTxvG+ntLG5tb1T3q3s7R8cVt2j446WmSK0TSSXqhdhTTkTtG2Y4bSXKoqTiNNuNLmZ+91HqjST4sFMUxokeCRYzAg2Vgrd6kDgiOMw787QndQ6dGte3VsArRO/IDUo0Ardr8FQkiyhwhCOte77XmqCHCvDCKezyiDTNMVkgke0b6nACdVBvjh8hs6tMkSxVLaEQQv190SOE62nSWQ7E2zGetWbi/95/czEjSBnIs0MFWS5KM44MhLNU0BDpigxfGoJJorZWxEZY4WJsVlVbAj+6svrpHNZ9726f39VazaKOMpwCmdwAT5cQxNuoQVtIJDBM7zCm/PkvDjvzseyteQUMyfwB87nD5gJkwA=</latexit><latexit sha1_base64="bhOtvWaf4NML8EAFWbV91QG2/q8=">AAAB+HicbVBNS8NAEJ3Ur1o/GvXoZbEInkoigj0WvHjwUMF+QBvCZrtpl252w+5GqKG/xIsHRbz6U7z5b9y2OWjrg4HHezPMzItSzrTxvG+ntLG5tb1T3q3s7R8cVt2j446WmSK0TSSXqhdhTTkTtG2Y4bSXKoqTiNNuNLmZ+91HqjST4sFMUxokeCRYzAg2Vgrd6kDgiOMw787QndQ6dGte3VsArRO/IDUo0Ardr8FQkiyhwhCOte77XmqCHCvDCKezyiDTNMVkgke0b6nACdVBvjh8hs6tMkSxVLaEQQv190SOE62nSWQ7E2zGetWbi/95/czEjSBnIs0MFWS5KM44MhLNU0BDpigxfGoJJorZWxEZY4WJsVlVbAj+6svrpHNZ9726f39VazaKOMpwCmdwAT5cQxNuoQVtIJDBM7zCm/PkvDjvzseyteQUMyfwB87nD5gJkwA=</latexit>

[1.0, 0, 0]Differentiable Model f(x, W) Pred Loss

rWLoss<latexit sha1_base64="bhOtvWaf4NML8EAFWbV91QG2/q8=">AAAB+HicbVBNS8NAEJ3Ur1o/GvXoZbEInkoigj0WvHjwUMF+QBvCZrtpl252w+5GqKG/xIsHRbz6U7z5b9y2OWjrg4HHezPMzItSzrTxvG+ntLG5tb1T3q3s7R8cVt2j446WmSK0TSSXqhdhTTkTtG2Y4bSXKoqTiNNuNLmZ+91HqjST4sFMUxokeCRYzAg2Vgrd6kDgiOMw787QndQ6dGte3VsArRO/IDUo0Ardr8FQkiyhwhCOte77XmqCHCvDCKezyiDTNMVkgke0b6nACdVBvjh8hs6tMkSxVLaEQQv190SOE62nSWQ7E2zGetWbi/95/czEjSBnIs0MFWS5KM44MhLNU0BDpigxfGoJJorZWxEZY4WJsVlVbAj+6svrpHNZ9726f39VazaKOMpwCmdwAT5cQxNuoQVtIJDBM7zCm/PkvDjvzseyteQUMyfwB87nD5gJkwA=</latexit><latexit sha1_base64="bhOtvWaf4NML8EAFWbV91QG2/q8=">AAAB+HicbVBNS8NAEJ3Ur1o/GvXoZbEInkoigj0WvHjwUMF+QBvCZrtpl252w+5GqKG/xIsHRbz6U7z5b9y2OWjrg4HHezPMzItSzrTxvG+ntLG5tb1T3q3s7R8cVt2j446WmSK0TSSXqhdhTTkTtG2Y4bSXKoqTiNNuNLmZ+91HqjST4sFMUxokeCRYzAg2Vgrd6kDgiOMw787QndQ6dGte3VsArRO/IDUo0Ardr8FQkiyhwhCOte77XmqCHCvDCKezyiDTNMVkgke0b6nACdVBvjh8hs6tMkSxVLaEQQv190SOE62nSWQ7E2zGetWbi/95/czEjSBnIs0MFWS5KM44MhLNU0BDpigxfGoJJorZWxEZY4WJsVlVbAj+6svrpHNZ9726f39VazaKOMpwCmdwAT5cQxNuoQVtIJDBM7zCm/PkvDjvzseyteQUMyfwB87nD5gJkwA=</latexit><latexit sha1_base64="bhOtvWaf4NML8EAFWbV91QG2/q8=">AAAB+HicbVBNS8NAEJ3Ur1o/GvXoZbEInkoigj0WvHjwUMF+QBvCZrtpl252w+5GqKG/xIsHRbz6U7z5b9y2OWjrg4HHezPMzItSzrTxvG+ntLG5tb1T3q3s7R8cVt2j446WmSK0TSSXqhdhTTkTtG2Y4bSXKoqTiNNuNLmZ+91HqjST4sFMUxokeCRYzAg2Vgrd6kDgiOMw787QndQ6dGte3VsArRO/IDUo0Ardr8FQkiyhwhCOte77XmqCHCvDCKezyiDTNMVkgke0b6nACdVBvjh8hs6tMkSxVLaEQQv190SOE62nSWQ7E2zGetWbi/95/czEjSBnIs0MFWS5KM44MhLNU0BDpigxfGoJJorZWxEZY4WJsVlVbAj+6svrpHNZ9726f39VazaKOMpwCmdwAT5cQxNuoQVtIJDBM7zCm/PkvDjvzseyteQUMyfwB87nD5gJkwA=</latexit><latexit sha1_base64="bhOtvWaf4NML8EAFWbV91QG2/q8=">AAAB+HicbVBNS8NAEJ3Ur1o/GvXoZbEInkoigj0WvHjwUMF+QBvCZrtpl252w+5GqKG/xIsHRbz6U7z5b9y2OWjrg4HHezPMzItSzrTxvG+ntLG5tb1T3q3s7R8cVt2j446WmSK0TSSXqhdhTTkTtG2Y4bSXKoqTiNNuNLmZ+91HqjST4sFMUxokeCRYzAg2Vgrd6kDgiOMw787QndQ6dGte3VsArRO/IDUo0Ardr8FQkiyhwhCOte77XmqCHCvDCKezyiDTNMVkgke0b6nACdVBvjh8hs6tMkSxVLaEQQv190SOE62nSWQ7E2zGetWbi/95/czEjSBnIs0MFWS5KM44MhLNU0BDpigxfGoJJorZWxEZY4WJsVlVbAj+6svrpHNZ9726f39VazaKOMpwCmdwAT5cQxNuoQVtIJDBM7zCm/PkvDjvzseyteQUMyfwB87nD5gJkwA=</latexit>

rWLoss0<latexit sha1_base64="lIuGqVxXUca2973zm1Y44JIhdMM=">AAAB+XicbVBNS8NAEJ3Ur1q/oh69LBbRU0lEsMeCFw8eKtgPaEPYbDft0s1u2N0USug/8eJBEa/+E2/+G7dtDtr6YODx3gwz86KUM20879spbWxube+Udyt7+weHR+7xSVvLTBHaIpJL1Y2wppwJ2jLMcNpNFcVJxGknGt/N/c6EKs2keDLTlAYJHgoWM4KNlULX7QsccRzmnRl6kFpfhm7Vq3kLoHXiF6QKBZqh+9UfSJIlVBjCsdY930tNkGNlGOF0VulnmqaYjPGQ9iwVOKE6yBeXz9CFVQYolsqWMGih/p7IcaL1NIlsZ4LNSK96c/E/r5eZuB7kTKSZoYIsF8UZR0aieQxowBQlhk8twUQxeysiI6wwMTasig3BX315nbSva75X8x9vqo16EUcZzuAcrsCHW2jAPTShBQQm8Ayv8Obkzovz7nwsW0tOMXMKf+B8/gD+DZMx</latexit><latexit sha1_base64="lIuGqVxXUca2973zm1Y44JIhdMM=">AAAB+XicbVBNS8NAEJ3Ur1q/oh69LBbRU0lEsMeCFw8eKtgPaEPYbDft0s1u2N0USug/8eJBEa/+E2/+G7dtDtr6YODx3gwz86KUM20879spbWxube+Udyt7+weHR+7xSVvLTBHaIpJL1Y2wppwJ2jLMcNpNFcVJxGknGt/N/c6EKs2keDLTlAYJHgoWM4KNlULX7QsccRzmnRl6kFpfhm7Vq3kLoHXiF6QKBZqh+9UfSJIlVBjCsdY930tNkGNlGOF0VulnmqaYjPGQ9iwVOKE6yBeXz9CFVQYolsqWMGih/p7IcaL1NIlsZ4LNSK96c/E/r5eZuB7kTKSZoYIsF8UZR0aieQxowBQlhk8twUQxeysiI6wwMTasig3BX315nbSva75X8x9vqo16EUcZzuAcrsCHW2jAPTShBQQm8Ayv8Obkzovz7nwsW0tOMXMKf+B8/gD+DZMx</latexit><latexit sha1_base64="lIuGqVxXUca2973zm1Y44JIhdMM=">AAAB+XicbVBNS8NAEJ3Ur1q/oh69LBbRU0lEsMeCFw8eKtgPaEPYbDft0s1u2N0USug/8eJBEa/+E2/+G7dtDtr6YODx3gwz86KUM20879spbWxube+Udyt7+weHR+7xSVvLTBHaIpJL1Y2wppwJ2jLMcNpNFcVJxGknGt/N/c6EKs2keDLTlAYJHgoWM4KNlULX7QsccRzmnRl6kFpfhm7Vq3kLoHXiF6QKBZqh+9UfSJIlVBjCsdY930tNkGNlGOF0VulnmqaYjPGQ9iwVOKE6yBeXz9CFVQYolsqWMGih/p7IcaL1NIlsZ4LNSK96c/E/r5eZuB7kTKSZoYIsF8UZR0aieQxowBQlhk8twUQxeysiI6wwMTasig3BX315nbSva75X8x9vqo16EUcZzuAcrsCHW2jAPTShBQQm8Ayv8Obkzovz7nwsW0tOMXMKf+B8/gD+DZMx</latexit><latexit sha1_base64="lIuGqVxXUca2973zm1Y44JIhdMM=">AAAB+XicbVBNS8NAEJ3Ur1q/oh69LBbRU0lEsMeCFw8eKtgPaEPYbDft0s1u2N0USug/8eJBEa/+E2/+G7dtDtr6YODx3gwz86KUM20879spbWxube+Udyt7+weHR+7xSVvLTBHaIpJL1Y2wppwJ2jLMcNpNFcVJxGknGt/N/c6EKs2keDLTlAYJHgoWM4KNlULX7QsccRzmnRl6kFpfhm7Vq3kLoHXiF6QKBZqh+9UfSJIlVBjCsdY930tNkGNlGOF0VulnmqaYjPGQ9iwVOKE6yBeXz9CFVQYolsqWMGih/p7IcaL1NIlsZ4LNSK96c/E/r5eZuB7kTKSZoYIsF8UZR0aieQxowBQlhk8twUQxeysiI6wwMTasig3BX315nbSva75X8x9vqo16EUcZzuAcrsCHW2jAPTShBQQm8Ayv8Obkzovz7nwsW0tOMXMKf+B8/gD+DZMx</latexit>

D = ||�W 0 ��W ||2<latexit sha1_base64="JDUXy6L/L94Jrb1LapMxrXDZo0k=">AAACEHicbZDLSsNAFIYn9VbrLerSzWCRurEkRbAboWAXLivYCzSxTKaTduhkEmYmQkn6CG58FTcuFHHr0p1v47SNoK0/DHz85xzmnN+LGJXKsr6M3Mrq2vpGfrOwtb2zu2fuH7RkGAtMmjhkoeh4SBJGOWkqqhjpRIKgwGOk7Y2upvX2PRGShvxWjSPiBmjAqU8xUtrqmSUnQGroeUl9Ai9hmjp1whSC7RI8gz+cpneVnlm0ytZMcBnsDIogU6Nnfjr9EMcB4QozJGXXtiLlJkgoihmZFJxYkgjhERqQrkaOAiLdZHbQBJ5opw/9UOjHFZy5vycSFEg5DjzdOV1fLtam5n+1bqz8qptQHsWKcDz/yI8ZVCGcpgP7VBCs2FgDwoLqXSEeIoGw0hkWdAj24snL0KqUbats35wXa9Usjjw4AsfgFNjgAtTANWiAJsDgATyBF/BqPBrPxpvxPm/NGdnMIfgj4+Mb94abOA==</latexit><latexit sha1_base64="JDUXy6L/L94Jrb1LapMxrXDZo0k=">AAACEHicbZDLSsNAFIYn9VbrLerSzWCRurEkRbAboWAXLivYCzSxTKaTduhkEmYmQkn6CG58FTcuFHHr0p1v47SNoK0/DHz85xzmnN+LGJXKsr6M3Mrq2vpGfrOwtb2zu2fuH7RkGAtMmjhkoeh4SBJGOWkqqhjpRIKgwGOk7Y2upvX2PRGShvxWjSPiBmjAqU8xUtrqmSUnQGroeUl9Ai9hmjp1whSC7RI8gz+cpneVnlm0ytZMcBnsDIogU6Nnfjr9EMcB4QozJGXXtiLlJkgoihmZFJxYkgjhERqQrkaOAiLdZHbQBJ5opw/9UOjHFZy5vycSFEg5DjzdOV1fLtam5n+1bqz8qptQHsWKcDz/yI8ZVCGcpgP7VBCs2FgDwoLqXSEeIoGw0hkWdAj24snL0KqUbats35wXa9Usjjw4AsfgFNjgAtTANWiAJsDgATyBF/BqPBrPxpvxPm/NGdnMIfgj4+Mb94abOA==</latexit><latexit sha1_base64="JDUXy6L/L94Jrb1LapMxrXDZo0k=">AAACEHicbZDLSsNAFIYn9VbrLerSzWCRurEkRbAboWAXLivYCzSxTKaTduhkEmYmQkn6CG58FTcuFHHr0p1v47SNoK0/DHz85xzmnN+LGJXKsr6M3Mrq2vpGfrOwtb2zu2fuH7RkGAtMmjhkoeh4SBJGOWkqqhjpRIKgwGOk7Y2upvX2PRGShvxWjSPiBmjAqU8xUtrqmSUnQGroeUl9Ai9hmjp1whSC7RI8gz+cpneVnlm0ytZMcBnsDIogU6Nnfjr9EMcB4QozJGXXtiLlJkgoihmZFJxYkgjhERqQrkaOAiLdZHbQBJ5opw/9UOjHFZy5vycSFEg5DjzdOV1fLtam5n+1bqz8qptQHsWKcDz/yI8ZVCGcpgP7VBCs2FgDwoLqXSEeIoGw0hkWdAj24snL0KqUbats35wXa9Usjjw4AsfgFNjgAtTANWiAJsDgATyBF/BqPBrPxpvxPm/NGdnMIfgj4+Mb94abOA==</latexit><latexit sha1_base64="hP+6LrUf2d3tZaldqaQQvEKMXyw=">AAAB2XicbZDNSgMxFIXv1L86Vq1rN8EiuCozbnQpuHFZwbZCO5RM5k4bmskMyR2hDH0BF25EfC93vo3pz0JbDwQ+zknIvSculLQUBN9ebWd3b/+gfugfNfzjk9Nmo2fz0gjsilzl5jnmFpXU2CVJCp8LgzyLFfbj6f0i77+gsTLXTzQrMMr4WMtUCk7O6oyaraAdLMW2IVxDC9YaNb+GSS7KDDUJxa0dhEFBUcUNSaFw7g9LiwUXUz7GgUPNM7RRtRxzzi6dk7A0N+5oYkv394uKZ9bOstjdzDhN7Ga2MP/LBiWlt1EldVESarH6KC0Vo5wtdmaJNChIzRxwYaSblYkJN1yQa8Z3HYSbG29D77odBu3wMYA6nMMFXEEIN3AHD9CBLghI4BXevYn35n2suqp569LO4I+8zx84xIo4</latexit><latexit sha1_base64="r1cJvcq9VF93K8CUdnoVf60bMk0=">AAACBXicbZDLSgMxGIX/qbdaq45u3QSL1I1lphvdCIJduKxgL9AZSybNtKGZC0lGKDN9BDe+ihsXivgK7nwbM20FbT0Q+DgnIf9/vJgzqSzryyisrW9sbhW3Szvl3b1986DcllEiCG2RiEei62FJOQtpSzHFaTcWFAcepx1vfJ3nnQcqJIvCOzWJqRvgYch8RrDSVt+sOgFWI89LG1N0ibLMaVCuMOpU0Rn64Sy7r/fNilWzZkKrYC+gAgs1++anM4hIEtBQEY6l7NlWrNwUC8UIp9OSk0gaYzLGQ9rTGOKASjedLTRFJ9oZID8S+oQKzdzfL1IcSDkJPH0zH18uZ7n5X9ZLlH/hpiyME0VDMv/ITzhSEcrbQQMmKFF8ogETwfSsiIywwETpDku6BHt55VVo12u2VbNvLSjCERzDKdhwDldwA01oAYFHeIZXeDOejBfjfV5XwVj0dgh/ZHx8A1AQmcc=</latexit><latexit sha1_base64="r1cJvcq9VF93K8CUdnoVf60bMk0=">AAACBXicbZDLSgMxGIX/qbdaq45u3QSL1I1lphvdCIJduKxgL9AZSybNtKGZC0lGKDN9BDe+ihsXivgK7nwbM20FbT0Q+DgnIf9/vJgzqSzryyisrW9sbhW3Szvl3b1986DcllEiCG2RiEei62FJOQtpSzHFaTcWFAcepx1vfJ3nnQcqJIvCOzWJqRvgYch8RrDSVt+sOgFWI89LG1N0ibLMaVCuMOpU0Rn64Sy7r/fNilWzZkKrYC+gAgs1++anM4hIEtBQEY6l7NlWrNwUC8UIp9OSk0gaYzLGQ9rTGOKASjedLTRFJ9oZID8S+oQKzdzfL1IcSDkJPH0zH18uZ7n5X9ZLlH/hpiyME0VDMv/ITzhSEcrbQQMmKFF8ogETwfSsiIywwETpDku6BHt55VVo12u2VbNvLSjCERzDKdhwDldwA01oAYFHeIZXeDOejBfjfV5XwVj0dgh/ZHx8A1AQmcc=</latexit><latexit sha1_base64="hvDkPWDSaNBgE/XR+Cd7hisrvZY=">AAACEHicbZDLSsNAFIYnXmu9RV26GSxSN5akG7sRCnbhsoK9QBPLZDpph04mYWYilKSP4MZXceNCEbcu3fk2TtoI2vrDwMd/zmHO+b2IUaks68tYWV1b39gsbBW3d3b39s2Dw7YMY4FJC4csFF0PScIoJy1FFSPdSBAUeIx0vPFVVu/cEyFpyG/VJCJugIac+hQjpa2+WXYCpEaelzSm8BKmqdMgTCHYKcNz+MNpelftmyWrYs0El8HOoQRyNfvmpzMIcRwQrjBDUvZsK1JugoSimJFp0YkliRAeoyHpaeQoINJNZgdN4al2BtAPhX5cwZn7eyJBgZSTwNOd2fpysZaZ/9V6sfJrbkJ5FCvC8fwjP2ZQhTBLBw6oIFixiQaEBdW7QjxCAmGlMyzqEOzFk5ehXa3YVsW+sUr1Wh5HARyDE3AGbHAB6uAaNEELYPAAnsALeDUejWfjzXift64Y+cwR+CPj4xv2Rps0</latexit><latexit sha1_base64="JDUXy6L/L94Jrb1LapMxrXDZo0k=">AAACEHicbZDLSsNAFIYn9VbrLerSzWCRurEkRbAboWAXLivYCzSxTKaTduhkEmYmQkn6CG58FTcuFHHr0p1v47SNoK0/DHz85xzmnN+LGJXKsr6M3Mrq2vpGfrOwtb2zu2fuH7RkGAtMmjhkoeh4SBJGOWkqqhjpRIKgwGOk7Y2upvX2PRGShvxWjSPiBmjAqU8xUtrqmSUnQGroeUl9Ai9hmjp1whSC7RI8gz+cpneVnlm0ytZMcBnsDIogU6Nnfjr9EMcB4QozJGXXtiLlJkgoihmZFJxYkgjhERqQrkaOAiLdZHbQBJ5opw/9UOjHFZy5vycSFEg5DjzdOV1fLtam5n+1bqz8qptQHsWKcDz/yI8ZVCGcpgP7VBCs2FgDwoLqXSEeIoGw0hkWdAj24snL0KqUbats35wXa9Usjjw4AsfgFNjgAtTANWiAJsDgATyBF/BqPBrPxpvxPm/NGdnMIfgj4+Mb94abOA==</latexit><latexit sha1_base64="JDUXy6L/L94Jrb1LapMxrXDZo0k=">AAACEHicbZDLSsNAFIYn9VbrLerSzWCRurEkRbAboWAXLivYCzSxTKaTduhkEmYmQkn6CG58FTcuFHHr0p1v47SNoK0/DHz85xzmnN+LGJXKsr6M3Mrq2vpGfrOwtb2zu2fuH7RkGAtMmjhkoeh4SBJGOWkqqhjpRIKgwGOk7Y2upvX2PRGShvxWjSPiBmjAqU8xUtrqmSUnQGroeUl9Ai9hmjp1whSC7RI8gz+cpneVnlm0ytZMcBnsDIogU6Nnfjr9EMcB4QozJGXXtiLlJkgoihmZFJxYkgjhERqQrkaOAiLdZHbQBJ5opw/9UOjHFZy5vycSFEg5DjzdOV1fLtam5n+1bqz8qptQHsWKcDz/yI8ZVCGcpgP7VBCs2FgDwoLqXSEeIoGw0hkWdAj24snL0KqUbats35wXa9Usjjw4AsfgFNjgAtTANWiAJsDgATyBF/BqPBrPxpvxPm/NGdnMIfgj4+Mb94abOA==</latexit><latexit sha1_base64="JDUXy6L/L94Jrb1LapMxrXDZo0k=">AAACEHicbZDLSsNAFIYn9VbrLerSzWCRurEkRbAboWAXLivYCzSxTKaTduhkEmYmQkn6CG58FTcuFHHr0p1v47SNoK0/DHz85xzmnN+LGJXKsr6M3Mrq2vpGfrOwtb2zu2fuH7RkGAtMmjhkoeh4SBJGOWkqqhjpRIKgwGOk7Y2upvX2PRGShvxWjSPiBmjAqU8xUtrqmSUnQGroeUl9Ai9hmjp1whSC7RI8gz+cpneVnlm0ytZMcBnsDIogU6Nnfjr9EMcB4QozJGXXtiLlJkgoihmZFJxYkgjhERqQrkaOAiLdZHbQBJ5opw/9UOjHFZy5vycSFEg5DjzdOV1fLtam5n+1bqz8qptQHsWKcDz/yI8ZVCGcpgP7VBCs2FgDwoLqXSEeIoGw0hkWdAj24snL0KqUbats35wXa9Usjjw4AsfgFNjgAtTANWiAJsDgATyBF/BqPBrPxpvxPm/NGdnMIfgj4+Mb94abOA==</latexit><latexit sha1_base64="JDUXy6L/L94Jrb1LapMxrXDZo0k=">AAACEHicbZDLSsNAFIYn9VbrLerSzWCRurEkRbAboWAXLivYCzSxTKaTduhkEmYmQkn6CG58FTcuFHHr0p1v47SNoK0/DHz85xzmnN+LGJXKsr6M3Mrq2vpGfrOwtb2zu2fuH7RkGAtMmjhkoeh4SBJGOWkqqhjpRIKgwGOk7Y2upvX2PRGShvxWjSPiBmjAqU8xUtrqmSUnQGroeUl9Ai9hmjp1whSC7RI8gz+cpneVnlm0ytZMcBnsDIogU6Nnfjr9EMcB4QozJGXXtiLlJkgoihmZFJxYkgjhERqQrkaOAiLdZHbQBJ5opw/9UOjHFZy5vycSFEg5DjzdOV1fLtam5n+1bqz8qptQHsWKcDz/yI8ZVCGcpgP7VBCs2FgDwoLqXSEeIoGw0hkWdAj24snL0KqUbats35wXa9Usjjw4AsfgFNjgAtTANWiAJsDgATyBF/BqPBrPxpvxPm/NGdnMIfgj4+Mb94abOA==</latexit><latexit sha1_base64="JDUXy6L/L94Jrb1LapMxrXDZo0k=">AAACEHicbZDLSsNAFIYn9VbrLerSzWCRurEkRbAboWAXLivYCzSxTKaTduhkEmYmQkn6CG58FTcuFHHr0p1v47SNoK0/DHz85xzmnN+LGJXKsr6M3Mrq2vpGfrOwtb2zu2fuH7RkGAtMmjhkoeh4SBJGOWkqqhjpRIKgwGOk7Y2upvX2PRGShvxWjSPiBmjAqU8xUtrqmSUnQGroeUl9Ai9hmjp1whSC7RI8gz+cpneVnlm0ytZMcBnsDIogU6Nnfjr9EMcB4QozJGXXtiLlJkgoihmZFJxYkgjhERqQrkaOAiLdZHbQBJ5opw/9UOjHFZy5vycSFEg5DjzdOV1fLtam5n+1bqz8qptQHsWKcDz/yI8ZVCGcpgP7VBCs2FgDwoLqXSEeIoGw0hkWdAj24snL0KqUbats35wXa9Usjjw4AsfgFNjgAtTANWiAJsDgATyBF/BqPBrPxpvxPm/NGdnMIfgj4+Mb94abOA==</latexit><latexit sha1_base64="hP+6LrUf2d3tZaldqaQQvEKMXyw=">AAAB2XicbZDNSgMxFIXv1L86Vq1rN8EiuCozbnQpuHFZwbZCO5RM5k4bmskMyR2hDH0BF25EfC93vo3pz0JbDwQ+zknIvSculLQUBN9ebWd3b/+gfugfNfzjk9Nmo2fz0gjsilzl5jnmFpXU2CVJCp8LgzyLFfbj6f0i77+gsTLXTzQrMMr4WMtUCk7O6oyaraAdLMW2IVxDC9YaNb+GSS7KDDUJxa0dhEFBUcUNSaFw7g9LiwUXUz7GgUPNM7RRtRxzzi6dk7A0N+5oYkv394uKZ9bOstjdzDhN7Ga2MP/LBiWlt1EldVESarH6KC0Vo5wtdmaJNChIzRxwYaSblYkJN1yQa8Z3HYSbG29D77odBu3wMYA6nMMFXEEIN3AHD9CBLghI4BXevYn35n2suqp569LO4I+8zx84xIo4</latexit><latexit sha1_base64="r1cJvcq9VF93K8CUdnoVf60bMk0=">AAACBXicbZDLSgMxGIX/qbdaq45u3QSL1I1lphvdCIJduKxgL9AZSybNtKGZC0lGKDN9BDe+ihsXivgK7nwbM20FbT0Q+DgnIf9/vJgzqSzryyisrW9sbhW3Szvl3b1986DcllEiCG2RiEei62FJOQtpSzHFaTcWFAcepx1vfJ3nnQcqJIvCOzWJqRvgYch8RrDSVt+sOgFWI89LG1N0ibLMaVCuMOpU0Rn64Sy7r/fNilWzZkKrYC+gAgs1++anM4hIEtBQEY6l7NlWrNwUC8UIp9OSk0gaYzLGQ9rTGOKASjedLTRFJ9oZID8S+oQKzdzfL1IcSDkJPH0zH18uZ7n5X9ZLlH/hpiyME0VDMv/ITzhSEcrbQQMmKFF8ogETwfSsiIywwETpDku6BHt55VVo12u2VbNvLSjCERzDKdhwDldwA01oAYFHeIZXeDOejBfjfV5XwVj0dgh/ZHx8A1AQmcc=</latexit><latexit sha1_base64="r1cJvcq9VF93K8CUdnoVf60bMk0=">AAACBXicbZDLSgMxGIX/qbdaq45u3QSL1I1lphvdCIJduKxgL9AZSybNtKGZC0lGKDN9BDe+ihsXivgK7nwbM20FbT0Q+DgnIf9/vJgzqSzryyisrW9sbhW3Szvl3b1986DcllEiCG2RiEei62FJOQtpSzHFaTcWFAcepx1vfJ3nnQcqJIvCOzWJqRvgYch8RrDSVt+sOgFWI89LG1N0ibLMaVCuMOpU0Rn64Sy7r/fNilWzZkKrYC+gAgs1++anM4hIEtBQEY6l7NlWrNwUC8UIp9OSk0gaYzLGQ9rTGOKASjedLTRFJ9oZID8S+oQKzdzfL1IcSDkJPH0zH18uZ7n5X9ZLlH/hpiyME0VDMv/ITzhSEcrbQQMmKFF8ogETwfSsiIywwETpDku6BHt55VVo12u2VbNvLSjCERzDKdhwDldwA01oAYFHeIZXeDOejBfjfV5XwVj0dgh/ZHx8A1AQmcc=</latexit><latexit sha1_base64="hvDkPWDSaNBgE/XR+Cd7hisrvZY=">AAACEHicbZDLSsNAFIYnXmu9RV26GSxSN5akG7sRCnbhsoK9QBPLZDpph04mYWYilKSP4MZXceNCEbcu3fk2TtoI2vrDwMd/zmHO+b2IUaks68tYWV1b39gsbBW3d3b39s2Dw7YMY4FJC4csFF0PScIoJy1FFSPdSBAUeIx0vPFVVu/cEyFpyG/VJCJugIac+hQjpa2+WXYCpEaelzSm8BKmqdMgTCHYKcNz+MNpelftmyWrYs0El8HOoQRyNfvmpzMIcRwQrjBDUvZsK1JugoSimJFp0YkliRAeoyHpaeQoINJNZgdN4al2BtAPhX5cwZn7eyJBgZSTwNOd2fpysZaZ/9V6sfJrbkJ5FCvC8fwjP2ZQhTBLBw6oIFixiQaEBdW7QjxCAmGlMyzqEOzFk5ehXa3YVsW+sUr1Wh5HARyDE3AGbHAB6uAaNEELYPAAnsALeDUejWfjzXift64Y+cwR+CPj4xv2Rps0</latexit><latexit sha1_base64="JDUXy6L/L94Jrb1LapMxrXDZo0k=">AAACEHicbZDLSsNAFIYn9VbrLerSzWCRurEkRbAboWAXLivYCzSxTKaTduhkEmYmQkn6CG58FTcuFHHr0p1v47SNoK0/DHz85xzmnN+LGJXKsr6M3Mrq2vpGfrOwtb2zu2fuH7RkGAtMmjhkoeh4SBJGOWkqqhjpRIKgwGOk7Y2upvX2PRGShvxWjSPiBmjAqU8xUtrqmSUnQGroeUl9Ai9hmjp1whSC7RI8gz+cpneVnlm0ytZMcBnsDIogU6Nnfjr9EMcB4QozJGXXtiLlJkgoihmZFJxYkgjhERqQrkaOAiLdZHbQBJ5opw/9UOjHFZy5vycSFEg5DjzdOV1fLtam5n+1bqz8qptQHsWKcDz/yI8ZVCGcpgP7VBCs2FgDwoLqXSEeIoGw0hkWdAj24snL0KqUbats35wXa9Usjjw4AsfgFNjgAtTANWiAJsDgATyBF/BqPBrPxpvxPm/NGdnMIfgj4+Mb94abOA==</latexit><latexit sha1_base64="JDUXy6L/L94Jrb1LapMxrXDZo0k=">AAACEHicbZDLSsNAFIYn9VbrLerSzWCRurEkRbAboWAXLivYCzSxTKaTduhkEmYmQkn6CG58FTcuFHHr0p1v47SNoK0/DHz85xzmnN+LGJXKsr6M3Mrq2vpGfrOwtb2zu2fuH7RkGAtMmjhkoeh4SBJGOWkqqhjpRIKgwGOk7Y2upvX2PRGShvxWjSPiBmjAqU8xUtrqmSUnQGroeUl9Ai9hmjp1whSC7RI8gz+cpneVnlm0ytZMcBnsDIogU6Nnfjr9EMcB4QozJGXXtiLlJkgoihmZFJxYkgjhERqQrkaOAiLdZHbQBJ5opw/9UOjHFZy5vycSFEg5DjzdOV1fLtam5n+1bqz8qptQHsWKcDz/yI8ZVCGcpgP7VBCs2FgDwoLqXSEeIoGw0hkWdAj24snL0KqUbats35wXa9Usjjw4AsfgFNjgAtTANWiAJsDgATyBF/BqPBrPxpvxPm/NGdnMIfgj4+Mb94abOA==</latexit><latexit sha1_base64="JDUXy6L/L94Jrb1LapMxrXDZo0k=">AAACEHicbZDLSsNAFIYn9VbrLerSzWCRurEkRbAboWAXLivYCzSxTKaTduhkEmYmQkn6CG58FTcuFHHr0p1v47SNoK0/DHz85xzmnN+LGJXKsr6M3Mrq2vpGfrOwtb2zu2fuH7RkGAtMmjhkoeh4SBJGOWkqqhjpRIKgwGOk7Y2upvX2PRGShvxWjSPiBmjAqU8xUtrqmSUnQGroeUl9Ai9hmjp1whSC7RI8gz+cpneVnlm0ytZMcBnsDIogU6Nnfjr9EMcB4QozJGXXtiLlJkgoihmZFJxYkgjhERqQrkaOAiLdZHbQBJ5opw/9UOjHFZy5vycSFEg5DjzdOV1fLtam5n+1bqz8qptQHsWKcDz/yI8ZVCGcpgP7VBCs2FgDwoLqXSEeIoGw0hkWdAj24snL0KqUbats35wXa9Usjjw4AsfgFNjgAtTANWiAJsDgATyBF/BqPBrPxpvxPm/NGdnMIfgj4+Mb94abOA==</latexit><latexit sha1_base64="JDUXy6L/L94Jrb1LapMxrXDZo0k=">AAACEHicbZDLSsNAFIYn9VbrLerSzWCRurEkRbAboWAXLivYCzSxTKaTduhkEmYmQkn6CG58FTcuFHHr0p1v47SNoK0/DHz85xzmnN+LGJXKsr6M3Mrq2vpGfrOwtb2zu2fuH7RkGAtMmjhkoeh4SBJGOWkqqhjpRIKgwGOk7Y2upvX2PRGShvxWjSPiBmjAqU8xUtrqmSUnQGroeUl9Ai9hmjp1whSC7RI8gz+cpneVnlm0ytZMcBnsDIogU6Nnfjr9EMcB4QozJGXXtiLlJkgoihmZFJxYkgjhERqQrkaOAiLdZHbQBJ5opw/9UOjHFZy5vycSFEg5DjzdOV1fLtam5n+1bqz8qptQHsWKcDz/yI8ZVCGcpgP7VBCs2FgDwoLqXSEeIoGw0hkWdAj24snL0KqUbats35wXa9Usjjw4AsfgFNjgAtTANWiAJsDgATyBF/BqPBrPxpvxPm/NGdnMIfgj4+Mb94abOA==</latexit><latexit sha1_base64="JDUXy6L/L94Jrb1LapMxrXDZo0k=">AAACEHicbZDLSsNAFIYn9VbrLerSzWCRurEkRbAboWAXLivYCzSxTKaTduhkEmYmQkn6CG58FTcuFHHr0p1v47SNoK0/DHz85xzmnN+LGJXKsr6M3Mrq2vpGfrOwtb2zu2fuH7RkGAtMmjhkoeh4SBJGOWkqqhjpRIKgwGOk7Y2upvX2PRGShvxWjSPiBmjAqU8xUtrqmSUnQGroeUl9Ai9hmjp1whSC7RI8gz+cpneVnlm0ytZMcBnsDIogU6Nnfjr9EMcB4QozJGXXtiLlJkgoihmZFJxYkgjhERqQrkaOAiLdZHbQBJ5opw/9UOjHFZy5vycSFEg5DjzdOV1fLtam5n+1bqz8qptQHsWKcDz/yI8ZVCGcpgP7VBCs2FgDwoLqXSEeIoGw0hkWdAj24snL0KqUbats35wXa9Usjjw4AsfgFNjgAtTANWiAJsDgATyBF/BqPBrPxpvxPm/NGdnMIfgj4+Mb94abOA==</latexit><latexit sha1_base64="JDUXy6L/L94Jrb1LapMxrXDZo0k=">AAACEHicbZDLSsNAFIYn9VbrLerSzWCRurEkRbAboWAXLivYCzSxTKaTduhkEmYmQkn6CG58FTcuFHHr0p1v47SNoK0/DHz85xzmnN+LGJXKsr6M3Mrq2vpGfrOwtb2zu2fuH7RkGAtMmjhkoeh4SBJGOWkqqhjpRIKgwGOk7Y2upvX2PRGShvxWjSPiBmjAqU8xUtrqmSUnQGroeUl9Ai9hmjp1whSC7RI8gz+cpneVnlm0ytZMcBnsDIogU6Nnfjr9EMcB4QozJGXXtiLlJkgoihmZFJxYkgjhERqQrkaOAiLdZHbQBJ5opw/9UOjHFZy5vycSFEg5DjzdOV1fLtam5n+1bqz8qptQHsWKcDz/yI8ZVCGcpgP7VBCs2FgDwoLqXSEeIoGw0hkWdAj24snL0KqUbats35wXa9Usjjw4AsfgFNjgAtTANWiAJsDgATyBF/BqPBrPxpvxPm/NGdnMIfgj4+Mb94abOA==</latexit>rXD

<latexit sha1_base64="4QmnqVAZoGZcfv4Vrd2WoALPJTw=">AAAB/nicbVDLSsNAFL2pr1pfUXHlZrAIrkoigl0WdOGygn1AE8JkOmmHTiZhZiKUUPBX3LhQxK3f4c6/cdJmoa0HBg7n3Ms9c8KUM6Ud59uqrK1vbG5Vt2s7u3v7B/bhUVclmSS0QxKeyH6IFeVM0I5mmtN+KimOQ0574eSm8HuPVCqWiAc9Takf45FgESNYGymwTzyBQ46DPkJejPU4DPPbGQrsutNw5kCrxC1JHUq0A/vLGyYki6nQhGOlBq6Taj/HUjPC6azmZYqmmEzwiA4MFTimys/n8Wfo3ChDFCXSPKHRXP29keNYqWkcmskiolr2CvE/b5DpqOnnTKSZpoIsDkUZRzpBRRdoyCQlmk8NwUQykxWRMZaYaNNYzZTgLn95lXQvG67TcO+v6q1mWUcVTuEMLsCFa2jBHbShAwRyeIZXeLOerBfr3fpYjFascucY/sD6/AEl75Tq</latexit><latexit sha1_base64="4QmnqVAZoGZcfv4Vrd2WoALPJTw=">AAAB/nicbVDLSsNAFL2pr1pfUXHlZrAIrkoigl0WdOGygn1AE8JkOmmHTiZhZiKUUPBX3LhQxK3f4c6/cdJmoa0HBg7n3Ms9c8KUM6Ud59uqrK1vbG5Vt2s7u3v7B/bhUVclmSS0QxKeyH6IFeVM0I5mmtN+KimOQ0574eSm8HuPVCqWiAc9Takf45FgESNYGymwTzyBQ46DPkJejPU4DPPbGQrsutNw5kCrxC1JHUq0A/vLGyYki6nQhGOlBq6Taj/HUjPC6azmZYqmmEzwiA4MFTimys/n8Wfo3ChDFCXSPKHRXP29keNYqWkcmskiolr2CvE/b5DpqOnnTKSZpoIsDkUZRzpBRRdoyCQlmk8NwUQykxWRMZaYaNNYzZTgLn95lXQvG67TcO+v6q1mWUcVTuEMLsCFa2jBHbShAwRyeIZXeLOerBfr3fpYjFascucY/sD6/AEl75Tq</latexit><latexit sha1_base64="4QmnqVAZoGZcfv4Vrd2WoALPJTw=">AAAB/nicbVDLSsNAFL2pr1pfUXHlZrAIrkoigl0WdOGygn1AE8JkOmmHTiZhZiKUUPBX3LhQxK3f4c6/cdJmoa0HBg7n3Ms9c8KUM6Ud59uqrK1vbG5Vt2s7u3v7B/bhUVclmSS0QxKeyH6IFeVM0I5mmtN+KimOQ0574eSm8HuPVCqWiAc9Takf45FgESNYGymwTzyBQ46DPkJejPU4DPPbGQrsutNw5kCrxC1JHUq0A/vLGyYki6nQhGOlBq6Taj/HUjPC6azmZYqmmEzwiA4MFTimys/n8Wfo3ChDFCXSPKHRXP29keNYqWkcmskiolr2CvE/b5DpqOnnTKSZpoIsDkUZRzpBRRdoyCQlmk8NwUQykxWRMZaYaNNYzZTgLn95lXQvG67TcO+v6q1mWUcVTuEMLsCFa2jBHbShAwRyeIZXeLOerBfr3fpYjFascucY/sD6/AEl75Tq</latexit><latexit sha1_base64="4QmnqVAZoGZcfv4Vrd2WoALPJTw=">AAAB/nicbVDLSsNAFL2pr1pfUXHlZrAIrkoigl0WdOGygn1AE8JkOmmHTiZhZiKUUPBX3LhQxK3f4c6/cdJmoa0HBg7n3Ms9c8KUM6Ud59uqrK1vbG5Vt2s7u3v7B/bhUVclmSS0QxKeyH6IFeVM0I5mmtN+KimOQ0574eSm8HuPVCqWiAc9Takf45FgESNYGymwTzyBQ46DPkJejPU4DPPbGQrsutNw5kCrxC1JHUq0A/vLGyYki6nQhGOlBq6Taj/HUjPC6azmZYqmmEzwiA4MFTimys/n8Wfo3ChDFCXSPKHRXP29keNYqWkcmskiolr2CvE/b5DpqOnnTKSZpoIsDkUZRzpBRRdoyCQlmk8NwUQykxWRMZaYaNNYzZTgLn95lXQvG67TcO+v6q1mWUcVTuEMLsCFa2jBHbShAwRyeIZXeLOerBfr3fpYjFascucY/sD6/AEl75Tq</latexit>

rY D<latexit sha1_base64="rSL6014zJx4Aw6fEEBNoQy0TIIk=">AAAB/nicbVDLSsNAFL3xWesrKq7cDBbBVUlEsMuCLlxWsA9pQphMJ+3QySTMTIQSCv6KGxeKuPU73Pk3TtostPXAwOGce7lnTphyprTjfFsrq2vrG5uVrer2zu7evn1w2FFJJgltk4QnshdiRTkTtK2Z5rSXSorjkNNuOL4u/O4jlYol4l5PUurHeChYxAjWRgrsY0/gkOPgASEvxnoUhvnNFAV2zak7M6Bl4pakBiVagf3lDRKSxVRowrFSfddJtZ9jqRnhdFr1MkVTTMZ4SPuGChxT5eez+FN0ZpQBihJpntBopv7eyHGs1CQOzWQRUS16hfif18901PBzJtJMU0Hmh6KMI52gogs0YJISzSeGYCKZyYrICEtMtGmsakpwF7+8TDoXddepu3eXtWajrKMCJ3AK5+DCFTThFlrQBgI5PMMrvFlP1ov1bn3MR1escucI/sD6/AEngJTr</latexit><latexit sha1_base64="rSL6014zJx4Aw6fEEBNoQy0TIIk=">AAAB/nicbVDLSsNAFL3xWesrKq7cDBbBVUlEsMuCLlxWsA9pQphMJ+3QySTMTIQSCv6KGxeKuPU73Pk3TtostPXAwOGce7lnTphyprTjfFsrq2vrG5uVrer2zu7evn1w2FFJJgltk4QnshdiRTkTtK2Z5rSXSorjkNNuOL4u/O4jlYol4l5PUurHeChYxAjWRgrsY0/gkOPgASEvxnoUhvnNFAV2zak7M6Bl4pakBiVagf3lDRKSxVRowrFSfddJtZ9jqRnhdFr1MkVTTMZ4SPuGChxT5eez+FN0ZpQBihJpntBopv7eyHGs1CQOzWQRUS16hfif18901PBzJtJMU0Hmh6KMI52gogs0YJISzSeGYCKZyYrICEtMtGmsakpwF7+8TDoXddepu3eXtWajrKMCJ3AK5+DCFTThFlrQBgI5PMMrvFlP1ov1bn3MR1escucI/sD6/AEngJTr</latexit><latexit sha1_base64="rSL6014zJx4Aw6fEEBNoQy0TIIk=">AAAB/nicbVDLSsNAFL3xWesrKq7cDBbBVUlEsMuCLlxWsA9pQphMJ+3QySTMTIQSCv6KGxeKuPU73Pk3TtostPXAwOGce7lnTphyprTjfFsrq2vrG5uVrer2zu7evn1w2FFJJgltk4QnshdiRTkTtK2Z5rSXSorjkNNuOL4u/O4jlYol4l5PUurHeChYxAjWRgrsY0/gkOPgASEvxnoUhvnNFAV2zak7M6Bl4pakBiVagf3lDRKSxVRowrFSfddJtZ9jqRnhdFr1MkVTTMZ4SPuGChxT5eez+FN0ZpQBihJpntBopv7eyHGs1CQOzWQRUS16hfif18901PBzJtJMU0Hmh6KMI52gogs0YJISzSeGYCKZyYrICEtMtGmsakpwF7+8TDoXddepu3eXtWajrKMCJ3AK5+DCFTThFlrQBgI5PMMrvFlP1ov1bn3MR1escucI/sD6/AEngJTr</latexit><latexit sha1_base64="rSL6014zJx4Aw6fEEBNoQy0TIIk=">AAAB/nicbVDLSsNAFL3xWesrKq7cDBbBVUlEsMuCLlxWsA9pQphMJ+3QySTMTIQSCv6KGxeKuPU73Pk3TtostPXAwOGce7lnTphyprTjfFsrq2vrG5uVrer2zu7evn1w2FFJJgltk4QnshdiRTkTtK2Z5rSXSorjkNNuOL4u/O4jlYol4l5PUurHeChYxAjWRgrsY0/gkOPgASEvxnoUhvnNFAV2zak7M6Bl4pakBiVagf3lDRKSxVRowrFSfddJtZ9jqRnhdFr1MkVTTMZ4SPuGChxT5eez+FN0ZpQBihJpntBopv7eyHGs1CQOzWQRUS16hfif18901PBzJtJMU0Hmh6KMI52gogs0YJISzSeGYCKZyYrICEtMtGmsakpwF7+8TDoXddepu3eXtWajrKMCJ3AK5+DCFTThFlrQBgI5PMMrvFlP1ov1bn3MR1escucI/sD6/AEngJTr</latexit>

Normal Client

Malicious Attacker

[0.2, 0.7, 0.1]Differentiable Model F(x’, W) Pred’ Loss’

[0, 1, 0]Differentiable Model F(x, W)

Pred Loss

Normal Participant

Malicious Attacker Try to match

Table 1

@D/@X<latexit sha1_base64="exZPuHJPrYbR0p/+Aezm6ffhGNM=">AAACCXicbVDLSsNAFL2pr1pfUZduBovgqiYi2GVBFy4r2Ac0oUymk3boZBJmJkIJ3brxV9y4UMStf+DOv3HSBtTWAwOHc+69c+8JEs6Udpwvq7Syura+Ud6sbG3v7O7Z+wdtFaeS0BaJeSy7AVaUM0FbmmlOu4mkOAo47QTjq9zv3FOpWCzu9CShfoSHgoWMYG2kvo28BEvNMEdehPUoCLLrKTr7Ubt9u+rUnBnQMnELUoUCzb796Q1ikkZUaMKxUj3XSbSf5fMIp9OKlyqaYDLGQ9ozVOCIKj+bXTJFJ0YZoDCW5gmNZurvjgxHSk2iwFTm66pFLxf/83qpDut+xkSSairI/KMw5UjHKI8FDZikRPOJIZhIZnZFZIQlJtqEVzEhuIsnL5P2ec11au7tRbVRL+IowxEcwym4cAkNuIEmtIDAAzzBC7xaj9az9Wa9z0tLVtFzCH9gfXwDet6ZgA==</latexit><latexit sha1_base64="exZPuHJPrYbR0p/+Aezm6ffhGNM=">AAACCXicbVDLSsNAFL2pr1pfUZduBovgqiYi2GVBFy4r2Ac0oUymk3boZBJmJkIJ3brxV9y4UMStf+DOv3HSBtTWAwOHc+69c+8JEs6Udpwvq7Syura+Ud6sbG3v7O7Z+wdtFaeS0BaJeSy7AVaUM0FbmmlOu4mkOAo47QTjq9zv3FOpWCzu9CShfoSHgoWMYG2kvo28BEvNMEdehPUoCLLrKTr7Ubt9u+rUnBnQMnELUoUCzb796Q1ikkZUaMKxUj3XSbSf5fMIp9OKlyqaYDLGQ9ozVOCIKj+bXTJFJ0YZoDCW5gmNZurvjgxHSk2iwFTm66pFLxf/83qpDut+xkSSairI/KMw5UjHKI8FDZikRPOJIZhIZnZFZIQlJtqEVzEhuIsnL5P2ec11au7tRbVRL+IowxEcwym4cAkNuIEmtIDAAzzBC7xaj9az9Wa9z0tLVtFzCH9gfXwDet6ZgA==</latexit><latexit sha1_base64="exZPuHJPrYbR0p/+Aezm6ffhGNM=">AAACCXicbVDLSsNAFL2pr1pfUZduBovgqiYi2GVBFy4r2Ac0oUymk3boZBJmJkIJ3brxV9y4UMStf+DOv3HSBtTWAwOHc+69c+8JEs6Udpwvq7Syura+Ud6sbG3v7O7Z+wdtFaeS0BaJeSy7AVaUM0FbmmlOu4mkOAo47QTjq9zv3FOpWCzu9CShfoSHgoWMYG2kvo28BEvNMEdehPUoCLLrKTr7Ubt9u+rUnBnQMnELUoUCzb796Q1ikkZUaMKxUj3XSbSf5fMIp9OKlyqaYDLGQ9ozVOCIKj+bXTJFJ0YZoDCW5gmNZurvjgxHSk2iwFTm66pFLxf/83qpDut+xkSSairI/KMw5UjHKI8FDZikRPOJIZhIZnZFZIQlJtqEVzEhuIsnL5P2ec11au7tRbVRL+IowxEcwym4cAkNuIEmtIDAAzzBC7xaj9az9Wa9z0tLVtFzCH9gfXwDet6ZgA==</latexit><latexit sha1_base64="exZPuHJPrYbR0p/+Aezm6ffhGNM=">AAACCXicbVDLSsNAFL2pr1pfUZduBovgqiYi2GVBFy4r2Ac0oUymk3boZBJmJkIJ3brxV9y4UMStf+DOv3HSBtTWAwOHc+69c+8JEs6Udpwvq7Syura+Ud6sbG3v7O7Z+wdtFaeS0BaJeSy7AVaUM0FbmmlOu4mkOAo47QTjq9zv3FOpWCzu9CShfoSHgoWMYG2kvo28BEvNMEdehPUoCLLrKTr7Ubt9u+rUnBnQMnELUoUCzb796Q1ikkZUaMKxUj3XSbSf5fMIp9OKlyqaYDLGQ9ozVOCIKj+bXTJFJ0YZoDCW5gmNZurvjgxHSk2iwFTm66pFLxf/83qpDut+xkSSairI/KMw5UjHKI8FDZikRPOJIZhIZnZFZIQlJtqEVzEhuIsnL5P2ec11au7tRbVRL+IowxEcwym4cAkNuIEmtIDAAzzBC7xaj9az9Wa9z0tLVtFzCH9gfXwDet6ZgA==</latexit>

@D/@Y<latexit sha1_base64="y0X4btE7egLo85xycGd8zg1c7+I=">AAACCXicbVDLSsNAFL3xWesr6tLNYBFc1UQEuyzowmUF+5AmlMl00g6dTMLMRCihWzf+ihsXirj1D9z5N07agNp6YOBwzr137j1BwpnSjvNlLS2vrK6tlzbKm1vbO7v23n5LxakktEliHstOgBXlTNCmZprTTiIpjgJO28HoMvfb91QqFotbPU6oH+GBYCEjWBupZyMvwVIzzJEXYT0Mguxqgk5/1LueXXGqzhRokbgFqUCBRs/+9PoxSSMqNOFYqa7rJNrP8nmE00nZSxVNMBnhAe0aKnBElZ9NL5mgY6P0URhL84RGU/V3R4YjpcZRYCrzddW8l4v/ed1UhzU/YyJJNRVk9lGYcqRjlMeC+kxSovnYEEwkM7siMsQSE23CK5sQ3PmTF0nrrOo6VffmvFKvFXGU4BCO4ARcuIA6XEMDmkDgAZ7gBV6tR+vZerPeZ6VLVtFzAH9gfXwDfGKZgQ==</latexit><latexit sha1_base64="y0X4btE7egLo85xycGd8zg1c7+I=">AAACCXicbVDLSsNAFL3xWesr6tLNYBFc1UQEuyzowmUF+5AmlMl00g6dTMLMRCihWzf+ihsXirj1D9z5N07agNp6YOBwzr137j1BwpnSjvNlLS2vrK6tlzbKm1vbO7v23n5LxakktEliHstOgBXlTNCmZprTTiIpjgJO28HoMvfb91QqFotbPU6oH+GBYCEjWBupZyMvwVIzzJEXYT0Mguxqgk5/1LueXXGqzhRokbgFqUCBRs/+9PoxSSMqNOFYqa7rJNrP8nmE00nZSxVNMBnhAe0aKnBElZ9NL5mgY6P0URhL84RGU/V3R4YjpcZRYCrzddW8l4v/ed1UhzU/YyJJNRVk9lGYcqRjlMeC+kxSovnYEEwkM7siMsQSE23CK5sQ3PmTF0nrrOo6VffmvFKvFXGU4BCO4ARcuIA6XEMDmkDgAZ7gBV6tR+vZerPeZ6VLVtFzAH9gfXwDfGKZgQ==</latexit><latexit sha1_base64="y0X4btE7egLo85xycGd8zg1c7+I=">AAACCXicbVDLSsNAFL3xWesr6tLNYBFc1UQEuyzowmUF+5AmlMl00g6dTMLMRCihWzf+ihsXirj1D9z5N07agNp6YOBwzr137j1BwpnSjvNlLS2vrK6tlzbKm1vbO7v23n5LxakktEliHstOgBXlTNCmZprTTiIpjgJO28HoMvfb91QqFotbPU6oH+GBYCEjWBupZyMvwVIzzJEXYT0Mguxqgk5/1LueXXGqzhRokbgFqUCBRs/+9PoxSSMqNOFYqa7rJNrP8nmE00nZSxVNMBnhAe0aKnBElZ9NL5mgY6P0URhL84RGU/V3R4YjpcZRYCrzddW8l4v/ed1UhzU/YyJJNRVk9lGYcqRjlMeC+kxSovnYEEwkM7siMsQSE23CK5sQ3PmTF0nrrOo6VffmvFKvFXGU4BCO4ARcuIA6XEMDmkDgAZ7gBV6tR+vZerPeZ6VLVtFzAH9gfXwDfGKZgQ==</latexit><latexit sha1_base64="y0X4btE7egLo85xycGd8zg1c7+I=">AAACCXicbVDLSsNAFL3xWesr6tLNYBFc1UQEuyzowmUF+5AmlMl00g6dTMLMRCihWzf+ihsXirj1D9z5N07agNp6YOBwzr137j1BwpnSjvNlLS2vrK6tlzbKm1vbO7v23n5LxakktEliHstOgBXlTNCmZprTTiIpjgJO28HoMvfb91QqFotbPU6oH+GBYCEjWBupZyMvwVIzzJEXYT0Mguxqgk5/1LueXXGqzhRokbgFqUCBRs/+9PoxSSMqNOFYqa7rJNrP8nmE00nZSxVNMBnhAe0aKnBElZ9NL5mgY6P0URhL84RGU/V3R4YjpcZRYCrzddW8l4v/ed1UhzU/YyJJNRVk9lGYcqRjlMeC+kxSovnYEEwkM7siMsQSE23CK5sQ3PmTF0nrrOo6VffmvFKvFXGU4BCO4ARcuIA6XEMDmkDgAZ7gBV6tR+vZerPeZ6VLVtFzAH9gfXwDfGKZgQ==</latexit>D = ||rW 0 �rW ||2

<latexit sha1_base64="ZHcrSD9GJX5ix03Zh0PiGAM2nZo=">AAACEnicbVDLSsNAFJ34rPUVdelmsIi6sCRFsBuhoAuXFewDmlgm00k7dDIJMxOhpPkGN/6KGxeKuHXlzr9x0kbQ1gMDh3Pu5c45XsSoVJb1ZSwsLi2vrBbWiusbm1vb5s5uU4axwKSBQxaKtockYZSThqKKkXYkCAo8Rlre8DLzW/dESBryWzWKiBugPqc+xUhpqWueOAFSA89LrlJ4AcdjhyOPIdg6gqfwh4/Hd0kl7Zolq2xNAOeJnZMSyFHvmp9OL8RxQLjCDEnZsa1IuQkSimJG0qITSxIhPER90tGUo4BIN5lESuGhVnrQD4V+XMGJ+nsjQYGUo8DTk1kAOetl4n9eJ1Z+1U0oj2JFOJ4e8mMGVQizfmCPCoIVG2mCsKD6rxAPkEBY6RaLugR7NvI8aVbKtlW2b85KtWpeRwHsgwNwDGxwDmrgGtRBA2DwAJ7AC3g1Ho1n4814n44uGPnOHvgD4+MbFOmcbA==</latexit><latexit sha1_base64="ZHcrSD9GJX5ix03Zh0PiGAM2nZo=">AAACEnicbVDLSsNAFJ34rPUVdelmsIi6sCRFsBuhoAuXFewDmlgm00k7dDIJMxOhpPkGN/6KGxeKuHXlzr9x0kbQ1gMDh3Pu5c45XsSoVJb1ZSwsLi2vrBbWiusbm1vb5s5uU4axwKSBQxaKtockYZSThqKKkXYkCAo8Rlre8DLzW/dESBryWzWKiBugPqc+xUhpqWueOAFSA89LrlJ4AcdjhyOPIdg6gqfwh4/Hd0kl7Zolq2xNAOeJnZMSyFHvmp9OL8RxQLjCDEnZsa1IuQkSimJG0qITSxIhPER90tGUo4BIN5lESuGhVnrQD4V+XMGJ+nsjQYGUo8DTk1kAOetl4n9eJ1Z+1U0oj2JFOJ4e8mMGVQizfmCPCoIVG2mCsKD6rxAPkEBY6RaLugR7NvI8aVbKtlW2b85KtWpeRwHsgwNwDGxwDmrgGtRBA2DwAJ7AC3g1Ho1n4814n44uGPnOHvgD4+MbFOmcbA==</latexit><latexit sha1_base64="ZHcrSD9GJX5ix03Zh0PiGAM2nZo=">AAACEnicbVDLSsNAFJ34rPUVdelmsIi6sCRFsBuhoAuXFewDmlgm00k7dDIJMxOhpPkGN/6KGxeKuHXlzr9x0kbQ1gMDh3Pu5c45XsSoVJb1ZSwsLi2vrBbWiusbm1vb5s5uU4axwKSBQxaKtockYZSThqKKkXYkCAo8Rlre8DLzW/dESBryWzWKiBugPqc+xUhpqWueOAFSA89LrlJ4AcdjhyOPIdg6gqfwh4/Hd0kl7Zolq2xNAOeJnZMSyFHvmp9OL8RxQLjCDEnZsa1IuQkSimJG0qITSxIhPER90tGUo4BIN5lESuGhVnrQD4V+XMGJ+nsjQYGUo8DTk1kAOetl4n9eJ1Z+1U0oj2JFOJ4e8mMGVQizfmCPCoIVG2mCsKD6rxAPkEBY6RaLugR7NvI8aVbKtlW2b85KtWpeRwHsgwNwDGxwDmrgGtRBA2DwAJ7AC3g1Ho1n4814n44uGPnOHvgD4+MbFOmcbA==</latexit><latexit sha1_base64="ZHcrSD9GJX5ix03Zh0PiGAM2nZo=">AAACEnicbVDLSsNAFJ34rPUVdelmsIi6sCRFsBuhoAuXFewDmlgm00k7dDIJMxOhpPkGN/6KGxeKuHXlzr9x0kbQ1gMDh3Pu5c45XsSoVJb1ZSwsLi2vrBbWiusbm1vb5s5uU4axwKSBQxaKtockYZSThqKKkXYkCAo8Rlre8DLzW/dESBryWzWKiBugPqc+xUhpqWueOAFSA89LrlJ4AcdjhyOPIdg6gqfwh4/Hd0kl7Zolq2xNAOeJnZMSyFHvmp9OL8RxQLjCDEnZsa1IuQkSimJG0qITSxIhPER90tGUo4BIN5lESuGhVnrQD4V+XMGJ+nsjQYGUo8DTk1kAOetl4n9eJ1Z+1U0oj2JFOJ4e8mMGVQizfmCPCoIVG2mCsKD6rxAPkEBY6RaLugR7NvI8aVbKtlW2b85KtWpeRwHsgwNwDGxwDmrgGtRBA2DwAJ7AC3g1Ho1n4814n44uGPnOHvgD4+MbFOmcbA==</latexit>

rW 0<latexit sha1_base64="Im5GdX2QSqf9bsuGMt3KJ1EK1Gw=">AAAB8HicdVDLSgMxFM3UV62vqks3wSK6GjKtj7oruHFZwT6kHUomzbShSWZIMkIZ+hVuXCji1s9x59+YaUdQ0QMXDufcy733BDFn2iD04RSWlldW14rrpY3Nre2d8u5eW0eJIrRFIh6pboA15UzSlmGG026sKBYBp51gcpX5nXuqNIvkrZnG1Bd4JFnICDZWuutLHHAMO8eDcgW5l8g7r3sQuWiOjJzVqtUa9HKlAnI0B+X3/jAiiaDSEI617nkoNn6KlWGE01mpn2gaYzLBI9qzVGJBtZ/OD57BI6sMYRgpW9LAufp9IsVC66kIbKfAZqx/e5n4l9dLTFj3UybjxFBJFovChEMTwex7OGSKEsOnlmCimL0VkjFWmBibUcmG8PUp/J+0q66HXO/mtNKo53EUwQE4BCfAAxegAa5BE7QAAQI8gCfw7Cjn0XlxXhetBSef2Qc/4Lx9Ahymj+o=</latexit><latexit sha1_base64="Im5GdX2QSqf9bsuGMt3KJ1EK1Gw=">AAAB8HicdVDLSgMxFM3UV62vqks3wSK6GjKtj7oruHFZwT6kHUomzbShSWZIMkIZ+hVuXCji1s9x59+YaUdQ0QMXDufcy733BDFn2iD04RSWlldW14rrpY3Nre2d8u5eW0eJIrRFIh6pboA15UzSlmGG026sKBYBp51gcpX5nXuqNIvkrZnG1Bd4JFnICDZWuutLHHAMO8eDcgW5l8g7r3sQuWiOjJzVqtUa9HKlAnI0B+X3/jAiiaDSEI617nkoNn6KlWGE01mpn2gaYzLBI9qzVGJBtZ/OD57BI6sMYRgpW9LAufp9IsVC66kIbKfAZqx/e5n4l9dLTFj3UybjxFBJFovChEMTwex7OGSKEsOnlmCimL0VkjFWmBibUcmG8PUp/J+0q66HXO/mtNKo53EUwQE4BCfAAxegAa5BE7QAAQI8gCfw7Cjn0XlxXhetBSef2Qc/4Lx9Ahymj+o=</latexit><latexit sha1_base64="Im5GdX2QSqf9bsuGMt3KJ1EK1Gw=">AAAB8HicdVDLSgMxFM3UV62vqks3wSK6GjKtj7oruHFZwT6kHUomzbShSWZIMkIZ+hVuXCji1s9x59+YaUdQ0QMXDufcy733BDFn2iD04RSWlldW14rrpY3Nre2d8u5eW0eJIrRFIh6pboA15UzSlmGG026sKBYBp51gcpX5nXuqNIvkrZnG1Bd4JFnICDZWuutLHHAMO8eDcgW5l8g7r3sQuWiOjJzVqtUa9HKlAnI0B+X3/jAiiaDSEI617nkoNn6KlWGE01mpn2gaYzLBI9qzVGJBtZ/OD57BI6sMYRgpW9LAufp9IsVC66kIbKfAZqx/e5n4l9dLTFj3UybjxFBJFovChEMTwex7OGSKEsOnlmCimL0VkjFWmBibUcmG8PUp/J+0q66HXO/mtNKo53EUwQE4BCfAAxegAa5BE7QAAQI8gCfw7Cjn0XlxXhetBSef2Qc/4Lx9Ahymj+o=</latexit><latexit sha1_base64="Im5GdX2QSqf9bsuGMt3KJ1EK1Gw=">AAAB8HicdVDLSgMxFM3UV62vqks3wSK6GjKtj7oruHFZwT6kHUomzbShSWZIMkIZ+hVuXCji1s9x59+YaUdQ0QMXDufcy733BDFn2iD04RSWlldW14rrpY3Nre2d8u5eW0eJIrRFIh6pboA15UzSlmGG026sKBYBp51gcpX5nXuqNIvkrZnG1Bd4JFnICDZWuutLHHAMO8eDcgW5l8g7r3sQuWiOjJzVqtUa9HKlAnI0B+X3/jAiiaDSEI617nkoNn6KlWGE01mpn2gaYzLBI9qzVGJBtZ/OD57BI6sMYRgpW9LAufp9IsVC66kIbKfAZqx/e5n4l9dLTFj3UybjxFBJFovChEMTwex7OGSKEsOnlmCimL0VkjFWmBibUcmG8PUp/J+0q66HXO/mtNKo53EUwQE4BCfAAxegAa5BE7QAAQI8gCfw7Cjn0XlxXhetBSef2Qc/4Lx9Ahymj+o=</latexit>

rW<latexit sha1_base64="rSfF52eD299KNblyESkZ7BtO9ek=">AAAB73icdVDLSgNBEOz1GeMr6tHLYBA8LbOJj3gLePEYwTwgWcLsZDYZMju7zswKIeQnvHhQxKu/482/cTZZQUULGoqqbrq7gkRwbTD+cJaWV1bX1gsbxc2t7Z3d0t5+S8epoqxJYxGrTkA0E1yypuFGsE6iGIkCwdrB+Crz2/dMaR7LWzNJmB+RoeQhp8RYqdOTJBAEtfulMnYvsXde8xB28RwZOatWKlXk5UoZcjT6pffeIKZpxKShgmjd9XBi/ClRhlPBZsVeqllC6JgMWddSSSKm/en83hk6tsoAhbGyJQ2aq98npiTSehIFtjMiZqR/e5n4l9dNTVjzp1wmqWGSLhaFqUAmRtnzaMAVo0ZMLCFUcXsroiOiCDU2oqIN4etT9D9pVVwPu97Nabley+MowCEcwQl4cAF1uIYGNIGCgAd4gmfnznl0XpzXReuSk88cwA84b5+5uI+5</latexit><latexit sha1_base64="rSfF52eD299KNblyESkZ7BtO9ek=">AAAB73icdVDLSgNBEOz1GeMr6tHLYBA8LbOJj3gLePEYwTwgWcLsZDYZMju7zswKIeQnvHhQxKu/482/cTZZQUULGoqqbrq7gkRwbTD+cJaWV1bX1gsbxc2t7Z3d0t5+S8epoqxJYxGrTkA0E1yypuFGsE6iGIkCwdrB+Crz2/dMaR7LWzNJmB+RoeQhp8RYqdOTJBAEtfulMnYvsXde8xB28RwZOatWKlXk5UoZcjT6pffeIKZpxKShgmjd9XBi/ClRhlPBZsVeqllC6JgMWddSSSKm/en83hk6tsoAhbGyJQ2aq98npiTSehIFtjMiZqR/e5n4l9dNTVjzp1wmqWGSLhaFqUAmRtnzaMAVo0ZMLCFUcXsroiOiCDU2oqIN4etT9D9pVVwPu97Nabley+MowCEcwQl4cAF1uIYGNIGCgAd4gmfnznl0XpzXReuSk88cwA84b5+5uI+5</latexit><latexit sha1_base64="rSfF52eD299KNblyESkZ7BtO9ek=">AAAB73icdVDLSgNBEOz1GeMr6tHLYBA8LbOJj3gLePEYwTwgWcLsZDYZMju7zswKIeQnvHhQxKu/482/cTZZQUULGoqqbrq7gkRwbTD+cJaWV1bX1gsbxc2t7Z3d0t5+S8epoqxJYxGrTkA0E1yypuFGsE6iGIkCwdrB+Crz2/dMaR7LWzNJmB+RoeQhp8RYqdOTJBAEtfulMnYvsXde8xB28RwZOatWKlXk5UoZcjT6pffeIKZpxKShgmjd9XBi/ClRhlPBZsVeqllC6JgMWddSSSKm/en83hk6tsoAhbGyJQ2aq98npiTSehIFtjMiZqR/e5n4l9dNTVjzp1wmqWGSLhaFqUAmRtnzaMAVo0ZMLCFUcXsroiOiCDU2oqIN4etT9D9pVVwPu97Nabley+MowCEcwQl4cAF1uIYGNIGCgAd4gmfnznl0XpzXReuSk88cwA84b5+5uI+5</latexit><latexit sha1_base64="rSfF52eD299KNblyESkZ7BtO9ek=">AAAB73icdVDLSgNBEOz1GeMr6tHLYBA8LbOJj3gLePEYwTwgWcLsZDYZMju7zswKIeQnvHhQxKu/482/cTZZQUULGoqqbrq7gkRwbTD+cJaWV1bX1gsbxc2t7Z3d0t5+S8epoqxJYxGrTkA0E1yypuFGsE6iGIkCwdrB+Crz2/dMaR7LWzNJmB+RoeQhp8RYqdOTJBAEtfulMnYvsXde8xB28RwZOatWKlXk5UoZcjT6pffeIKZpxKShgmjd9XBi/ClRhlPBZsVeqllC6JgMWddSSSKm/en83hk6tsoAhbGyJQ2aq98npiTSehIFtjMiZqR/e5n4l9dNTVjzp1wmqWGSLhaFqUAmRtnzaMAVo0ZMLCFUcXsroiOiCDU2oqIN4etT9D9pVVwPu97Nabley+MowCEcwQl4cAF1uIYGNIGCgAd4gmfnznl0XpzXReuSk88cwA84b5+5uI+5</latexit>

Figure 2: The overview of our DLG algorithm. Variables to be updated are marked with a boldborder. While normal participants calculate ∇W to update parameter using its private training data,the malicious attacker updates its dummy inputs and labels to minimize the gradients distance. Whenthe optimization finishes, the evil user is able to obtain the training set from honest participants.

Algorithm 1 Deep Leakage from Gradients.Input: F (x;W ): Differentiable machine learning model; W : parameter weights; ∇W : gradi-

ents calculated by training dataOutput: private training data x,y

1: procedure DLG(F , W ,∇W )2: x′1 ← N (0, 1) , y′1 ← N (0, 1) . Initialize dummy inputs and labels.3: for i← 1 to n do4: ∇W ′i ← ∂`(F (x′i,Wt),y

′i)/∂Wt . Compute dummy gradients.

5: Di ← ||∇W ′i −∇W ||26: x′i+1 ← x′i − η∇x′

iDi , y′i+1 ← y′i − η∇y′

iDi . Update data to match gradients.

7: end for8: return x′n+1,y

′n+1

9: end procedure

3.1 Training Data Leakage through Gradients Matching

To recover the data from gradients, we first randomly initialize a dummy input x′ and label input y′.We then feed these “dummy data” into models and get “dummy gradients”.

∇W ′ = ∂`(F (x′,W ),y′)

∂W(3)

Optimizing the dummy gradients close as to original also makes the dummy data close to the realtraining data (the trends shown in Fig. 5). Given gradients at a certain step, we obtain the trainingdata by minimizing the following objective

x′∗,y′∗= argmin

x′,y′||∇W ′ −∇W ||2 = argmin

x′,y′||∂`(F (x

′,W ),y′)

∂W−∇W ||2 (4)

The distance ||∇W ′ −∇W ||2 is differentiable w.r.t dummy inputs x′ and labels y′ can thus can beoptimized using standard gradient-based methods. Note that this optimization requires 2nd orderderivatives. We make a mild assumption that F is twice differentiable, which holds for the majorityof modern machine learning models (e.g., most neural networks) and tasks.

4

Iters=0 Iters=10 Iters=50 Iters=100 Iters=500 Melis [27] Ground Truth

Figure 3: The visualization showing the deep leakage on images from MNIST [22], CIFAR-100 [21],SVHN [28] and LFW [14] respectively. Our algorithm fully recovers the four images while previouswork only succeeds on simple images with clean backgrounds.

3.2 Deep Leakage for Batched Data

The algo. 1 works well when there is only a single pair of input and label in the batch. However whenwe naively apply it to the case where batch size N ≥ 1, the algorithm would be too slow to converge.We think the reason is that batched data can have N ! different permutations and thus make optimizerhard to choose gradient directions. To force the optimization closer to a solution, instead of updatingthe whole batch, we update a single training sample instead. We modify the line 6 in algo. 1 to :

x′i mod Nt+1 ← x′

i mod Nt −∇x′i mod N

t+1D

y′i mod Nt+1 ← y′

i mod Nt −∇y′i mod N

t+1D

(5)

Then we can observe fast and stable convergence. We list the iterations required for convergence fordifferent batch sizes in Tab. 1 and provide visualized results in Fig. 4. The larger the batch size is, themore iterations DLG requires to attack.

BS=1 BS=2 BS=4 BS=8ResNet-20 270 602 1173 2711

Table 1: The iterations required for restore batched data on CIFAR [21] dataset.

Initial

Middle Stage

Fully Leaked

Ground Truth

Figure 4: Results of deep leakage of batched data. Though the order may not be the same and thereare more artifact pixels, DLG still produces images very close to the original ones.

5

Mea

n Sq

uare

Erro

r

0.0

0.1

0.2

0.2

0.3

MNIST CIFAR SVHN LFWMelis Ours

Table 1

MNIST CIFAR SVHN LFW

Ours 0.0038 0.0069 0.0051 0.0055

Melis 0.2275 0.2578 0.2771 0.2951

Figure 5: Layer-i means MSE between real anddummy gradients of ith layer. When the gradients’distance gets smaller, the MSE between leakedimage and the original image also gets smaller.

Mea

n Sq

uare

Erro

r

0.0

0.1

0.2

0.2

0.3

MNIST CIFAR SVHN LFWMelis Ours

Table 1

MNIST CIFAR SVHN LFW

Ours 0.0038 0.0069 0.0051 0.0055

Melis 0.2275 0.2578 0.2771 0.2951

Figure 6: Compassion of the MSE of imagesleaked by different algorithms and the groundtruth. Our method consistently outperformsprevious approach by a large margin.

4 Experiments

Setup. Implementing algorithm. 1 requires to calculate the high order gradients and we choosePyTorch [29] as our experiment platform. We use L-BFGS [25] with learning rate 1, history size100 and max iterations 20 and optimize for 1200 iterations and 100 iterations for image and texttask respectively. We aim to match gradients from all trainable parameters. Notably, DLG has norequirements on model’s convergence status, in another word, the attack can happen anytime duringthe training. To be more general, all our experiments are using randomly initialized weights. Moretask-specific details can be found in following sub-sections.

4.1 Deep Leakage on Image Classification

Given an image containing objects, images classification aims to determine the class of the item.We experiment our algorithm on modern CNN architectures ResNet-56 [12] and pictures fromMNIST [22], CIFAR-100 [21], SVHN [28] and LFW [14]. Two changes we have made to the modelsare replacing activation ReLU to Sigmoid and removing strides, as our algorithm requires the modelto be twice-differentiable. For image labels, instead of directly optimizing the discrete categoricalvalues, we random initialize a vector with shape N × C where N is the batch size and C is thenumber of classes, and then take its softmax output as the one-hot label for optimization.

The leaking process is visualized in Fig. 3. We start with random Gaussian noise (first column) andtry to match the gradients produced by the dummy data and real ones. As shown in Fig 5, minimizingthe distance between gradients also reduces the gap between data. We observe that monochromeimages with a clean background (MNIST) are easiest to recover, while complex images like face takemore iterations to recover (Fig. 3). When the optimization finishes, the recover results are almostidentical to ground truth images, despite few negligible artifact pixels.

We visually compare the results from other method [27] and ours in Fig. 3. The previous method usesGAN models when the class label is given and only works well on MNIST. The result on SVHN,though is still visually recognizable as digit “9”, this is no longer the original training image. Thecases are even worse on LFW and collapse on CIFAR. We also make a numerical comparison byperforming leaking and measuring the MSE on all dataset images in Fig. 6. Images are normalized tothe range [0, 1] and our algorithm appears much better results (ours < 0.03 v.s. previous > 0.2) onall four datasets.

4.2 Deep Leakage on Masked Language Model

For language task, we verify our algorithm on Masked Language Model (MLM) task. In eachsequence, 15% of the words are replaced with a [MASK] token and MLM model attempts to predict

6

Example 1 Example 2 Example 3Initial Sen-tence

tilting fill given **less word**itude fine **nton over-heard living vegas **vac**vation *f forte **dis ce-rambycidae ellison **donyards marne **kali

toni **enting asbestos cut-ler km nail **oof **dation**ori righteous **xie lucan**hot **ery at **tle orderedpa **eit smashing proto

[MASK] **ry toppled**wled major relief divedisplaced **lice [CLS] usapps _ **face **bet

Iters = 10 tilting fill given **less fullsolicitor other ligue shrillliving vegas rider treatmentcarry played sculptures life-long ellison net yards marne**kali

toni **enting asbestos cutterkm nail undefeated **dationhole righteous **xie lucan**hot **ery at **tle orderedpa **eit smashing proto

[MASK] **ry toppled iden-tified major relief gin divedisplaced **lice doll usapps _ **face space

Iters = 20 registration , volunteer ap-plications , at student travelapplication open the ; weekof played ; child care will beglare .

we welcome proposals fortutor **ials on either coremachine denver softly ortopics of emerging impor-tance for machine learning.

one **ry toppled hold majorritual ’ dive annual confer-ence days 1924 apps novel-ist dude space

Iters = 30 registration , volunteer ap-plications , and studenttravel application open thefirst week of september .child care will be available .

we welcome proposals fortutor **ials on either coremachine learning topics ortopics of emerging impor-tance for machine learning.

we invite submissions forthe thirty - third annual con-ference on neural informa-tion processing systems .

OriginalText

Registration, volunteerapplications, and studenttravel application open thefirst week of September.Child care will be available.

We welcome proposals fortutorials on either core ma-chine learning topics or top-ics of emerging importancefor machine learning.

We invite submissions forthe Thirty-Third AnnualConference on Neural Infor-mation Processing Systems.

Table 2: The progress of deep leakage on language tasks.

the original value of the masked words from a given context. We choose BERT [8] as our backboneand adapt hyperparameters from the official implementation ∗.

Different from vision tasks where RGB inputs are continuous values, language models need topreprocess discrete words into embeddings. We apply DLG on embedding space and minimize thegradients distance between dummy embeddings and real ones. After optimization finishes, we deriveoriginal words by finding the closest entry in the embedding matrix reversely.

In Tab. 2, we exhibit the leaking history on three sentences selected from NeurIPS conference page.Similar to the vision task, we start with randomly initialized embedding: the reverse query results atiteration 0 is meaningless. During the optimization, the gradients produced by dummy embeddinggradually match the original ones and so the embeddings. In later iterations, part of sequencegradually appears. In example 3, at iteration 20, ‘annual conference’ appeared and at iteration 30 andthe leaked sentence is already close to the original one. When DLG finishes, though there are fewmismatches caused by the ambiguity in tokenizing, the main content is already fully leaked.

5 Defense Strategies

5.1 Noisy Gradients

One straightforward attempt to defense DLG is to add noise on gradients before sharing. To evaluate,we experiment Gaussian and Laplacian noise (widely used in differential privacy studies) distributionswith variance range from 10−1 to 10−4 and central 0. From Fig. 7a and 7b, we observe that thedefense effect mainly depends on the magnitude of distribution variance and less related to the noisetypes. When variance is at the scale of 10−4, the noisy gradients do not prevent the leak. For noisewith variance 10−3, though with artifacts, the leakage can still be performed. Only when the varianceis larger than 10−2 and the noise is starting affect the accuracy, DLG will fail to execute and Laplacian

∗https://github.com/google-research/bert

7

https://github.com/google-research/bert

0 200 400 600 800 1000 1200Iterations

0.000

0.025

0.050

0.075

0.100

0.125

0.150

Gra

dien

tM

atch

Los

s

original

gaussian-10�4

gaussian-10�3

gaussian-10�2

gaussian-10�1Deep Leakage

Leak with artifacts

No leak

0 200 400 600 800 1000 1200Iterations

0.000

0.025

0.050

0.075

0.100

0.125

0.150

Gra

dien

tM

atch

Los

s

original

laplacian-10�4

laplacian-10�3

laplacian-10�2

laplacian-10�1 Deep Leakage

Leak with artifacts

No leak

0 200 400 600 800 1000 1200Iterations

0.000

0.025

0.050

0.075

0.100

0.125

0.150

Gra

dien

tM

atch

Los

s

original

IEEE-fp16

B-fp16Deep Leakage

0 200 400 600 800 1000 1200Iterations

0.000

0.025

0.050

0.075

0.100

0.125

0.150

Gra

dien

tM

atch

Los

s

original

prune-ratio-1%

prune-ratio-10%

prune-ratio-20%

prune-ratio-30%

prune-ratio-50%

prune-ratio-70% Deep Leakage

Leak with artifacts

No leak

(a) Defend with different magnitude Gaussian noise.

0 200 400 600 800 1000 1200Iterations

0.000

0.025

0.050

0.075

0.100

0.125

0.150

Gra

dien

tM

atch

Los

s

original

gaussian-10�4

gaussian-10�3

gaussian-10�2


Leak with artifacts

No leak

0 200 400 600 800 1000 1200Iterations

0.000

0.025

0.050

0.075

0.100

0.125

0.150

Gra

dien

tM

atch

Los

s

original

laplacian-10�4

laplacian-10�3

laplacian-10�2


Leak with artifacts

No leak

0 200 400 600 800 1000 1200Iterations

0.000

0.025

0.050

0.075

0.100

0.125

0.150

Gra

dien

tM

atch

Los

s

original

IEEE-fp16

B-fp16Deep Leakage

0 200 400 600 800 1000 1200Iterations

0.000

0.025

0.050

0.075

0.100

0.125

0.150

Gra

dien

tM

atch

Los

s

original

prune-ratio-1%

prune-ratio-10%

prune-ratio-20%

prune-ratio-30%

prune-ratio-50%


Leak with artifacts

No leak

(b) Defend with different magnitude Laplacian noise.

0 200 400 600 800 1000 1200Iterations

0.000

0.025

0.050

0.075

0.100

0.125

0.150

Gra

dien

tM

atch

Los

s

original

gaussian-10�4

gaussian-10�3

gaussian-10�2


Leak with artifacts

No leak

0 200 400 600 800 1000 1200Iterations

0.000

0.025

0.050

0.075

0.100

0.125

0.150

Gra

dien

tM

atch

Los

s

original

laplacian-10�4

laplacian-10�3

laplacian-10�2


Leak with artifacts

No leak

0 200 400 600 800 1000 1200Iterations

0.000

0.025

0.050

0.075

0.100

0.125

0.150

Gra

dien

tM

atch

Los

s

original

IEEE-fp16

B-fp16Deep Leakage

0 200 400 600 800 1000 1200Iterations

0.000

0.025

0.050

0.075

0.100

0.125

0.150

Gra

dien

tM

atch

Los

s

original

prune-ratio-1%

prune-ratio-10%

prune-ratio-20%

prune-ratio-30%

prune-ratio-50%


Leak with artifacts

No leak

(c) Defend with fp16 convertion.

0 200 400 600 800 1000 1200Iterations

0.000

0.025

0.050

0.075

0.100

0.125

0.150

Gra

dien

tM

atch

Los

s

original

gaussian-10�4

gaussian-10�3

gaussian-10�2


Leak with artifacts

No leak

0 200 400 600 800 1000 1200Iterations

0.000

0.025

0.050

0.075

0.100

0.125

0.150

Gra

dien

tM

atch

Los

s

original

laplacian-10�4

laplacian-10�3

laplacian-10�2


Leak with artifacts

No leak

0 200 400 600 800 1000 1200Iterations

0.000

0.025

0.050

0.075

0.100

0.125

0.150

Gra

dien

tM

atch

Los

s

original

IEEE-fp16

B-fp16Deep Leakage

0 200 400 600 800 1000 1200Iterations

0.000

0.025

0.050

0.075

0.100

0.125

0.150

Gra

dien

tM

atch

Los

s

original

prune-ratio-1%

prune-ratio-10%

prune-ratio-20%

prune-ratio-30%

prune-ratio-50%


Leak with artifacts

No leak

(d) Defend with gradient pruning.

Figure 7: The effectiveness of various defense strategies.

Original G-10−4 G-10−3 G-10−2 G-10−1 FP-16Accuracy 76.3% 75.6% 73.3% 45.3% ≤1% 76.1%

Defendability – 7 7 3 3 7

L-10−4 L-10−3 L-10−2 L-10−1 Int-8Accuracy – 75.6% 73.4% 46.2% ≤1% 53.7%

Defendability – 7 7 3 3 3Table 3: The trade-off between accuracy and defendability. G: Gaussian noise, L: Laplacian noise,FP: Floating number, Int: Integer quantization. 3 means it successfully defends against DLG while7 means fails to defend (whether the results are visually recognizable). The accuracy is evaluated onCIFAR-100.

tends to slight better at scale 10−3. However, noise with variance larger than 10−2 will degrade theaccuracy significantly (Tab. 3).

Another common perturbation on gradients is half precision, which was initially designed to saveGPU memory footprints and also widely used to reduce communication bandwidth. We test twopopular half precision implementations IEEE float16 (Single-precision floating-point format) andbfloat16 (Brain Floating Point [35], a truncated version of 32 bit float). Shown in Fig. 7c, both halfprecision formats fail to protect the training data. We also test another popular low-bit representationInt-8. Though it successfully prevents the leakage, the performance of model drops seriously (Tab. 3).

5.2 Gradient Compression and Sparsification

We next experimented to defend by gradient compression [24, 36]: Gradients with small magnitudesare pruned to zero. It’s more difficult for DLG to match the gradients as the optimization targets arepruned. We evaluate how different level of sparsities (range from 1% to 70%) defense the leakage.When sparsity is 1% to 10%, it has almost no effects against DLG. When prune ratio increases to20%, as shown in Fig. 7d, there are obvious artifact pixels on the recover images. We notice that

8

maximum tolerance of sparsity is around 20%. When pruning ratio is larger, the recovered imagesare no longer visually recognizable and thus gradient compression successfully prevents the leakage.

Previous work [24, 36] show that gradients can be compressed by more than 300× without losingaccuracy by error compensation techniques. In this case, the sparsity is above 99% and alreadyexceeds the maximum tolerance of DLG (which is around 20%). It suggests that compressing thegradients is a practical approach to avoid the deep leakage.

5.3 Large Batch, High Resolution and Cryptology

If changes on training settings are allowed, then there are more defense strategies. As suggested inTab. 1, increasing the batch size makes the leakage more difficult because there are more variablesto solve during optimization. Following the idea, upscaling the input images can be also a gooddefense, though some changes on CNN architectures is required. According to our experiments, DLGcurrently only works for batch size up to 8 and image resolution up to 64×64.

Beside methods above, cryptology can be also used to prevent the leakage: Bonawitz et al. [5]designs a secure aggregation protocols and Phong et al. [31] proposes to encrypt the gradients beforesending. Among all defenses, cryptology is the most secured one. However, both methods have theirlimitations and not general enough: secure aggregation [5] requires gradients to be integers thus notcompatible with most CNNs, and homomorphic encryption [31] is against parameter server only.

6 Conclusions

In this paper, we introduce the Deep Leakage from Gradients (DLG): an algorithm that can obtain thelocal training data from public shared gradients. DLG does not rely on any generative model or extraprior about the data. Our experiments on vision and language tasks both demonstrate the criticalrisks of such deep leakage and show that such deep leakage can be only prevented when defensestrategies starts to degrade the accuracy. This sets a challenge to modern multi-node learning system(e.g., distributed training, federated learning). We hope this work would raise people’s awarenessabout the security of gradients and bring the community to rethink the safety of existing gradientsharing scheme.

Acknowledgments

We sincerely thank MIT-IBM Watson AI lab, Intel, Facebook and AWS for supporting this work. Wesincerely thank John Cohn for the discussions.

References

[1] Martín Abadi, Ashish Agarwal, Paul Barham, Eugene Brevdo, Zhifeng Chen, Craig Citro, Greg S. Corrado,Andy Davis, Jeffrey Dean, Matthieu Devin, Sanjay Ghemawat, Ian Goodfellow, Andrew Harp, GeoffreyIrving, Michael Isard, Yangqing Jia, Rafal Jozefowicz, Lukasz Kaiser, Manjunath Kudlur, Josh Levenberg,Dan Mané, Rajat Monga, Sherry Moore, Derek Murray, Chris Olah, Mike Schuster, Jonathon Shlens,Benoit Steiner, Ilya Sutskever, Kunal Talwar, Paul Tucker, Vincent Vanhoucke, Vijay Vasudevan, FernandaViégas, Oriol Vinyals, Pete Warden, Martin Wattenberg, Martin Wicke, Yuan Yu, and Xiaoqiang Zheng.TensorFlow: Large-scale machine learning on heterogeneous systems, 2015. Software available fromtensorflow.org. 3

[2] Takuya Akiba, Keisuke Fukuda, and Shuji Suzuki. ChainerMN: Scalable Distributed Deep LearningFramework. In Proceedings of Workshop on ML Systems in The Thirty-first Annual Conference on NeuralInformation Processing Systems (NIPS), 2017. 3

[3] Blaise Barney et al. Introduction to parallel computing. 1, 3[4] Keith Bonawitz, Hubert Eichner, Wolfgang Grieskamp, Dzmitry Huba, Alex Ingerman, Vladimir Ivanov,

Chloe Kiddon, Jakub Konecny, Stefano Mazzocchi, H Brendan McMahan, et al. Towards federatedlearning at scale: System design. arXiv preprint arXiv:1902.01046, 2019. 3

[5] Keith Bonawitz, Vladimir Ivanov, Ben Kreuter, Antonio Marcedone, H. Brendan McMahan, Sarvar Patel,Daniel Ramage, Aaron Segal, and Karn Seth. Practical secure aggregation for federated learning onuser-held data. CoRR, abs/1611.04482, 2016. 9

9

[6] Tianqi Chen, Mu Li, Yutian Li, Min Lin, Naiyan Wang, Minjie Wang, Tianjun Xiao, Bing Xu, ChiyuanZhang, and Zheng Zhang. Mxnet: A flexible and efficient machine learning library for heterogeneousdistributed systems. arXiv preprint arXiv:1512.01274, 2015. 3

[7] Jeffrey Dean, Greg Corrado, Rajat Monga, Kai Chen, Matthieu Devin, Mark Mao, Andrew Senior, PaulTucker, Ke Yang, Quoc V Le, et al. Large scale distributed deep networks. In Advances in neuralinformation processing systems, pages 1223–1231, 2012. 3

[8] Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. BERT: pre-training of deepbidirectional transformers for language understanding. CoRR, abs/1810.04805, 2018. 7

[9] Matt Fredrikson, Somesh Jha, and Thomas Ristenpart. Model inversion attacks that exploit confidenceinformation and basic countermeasures. In Proceedings of the 22nd ACM SIGSAC Conference on Computerand Communications Security, pages 1322–1333. ACM, 2015. 1, 3

[10] Ian Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, AaronCourville, and Yoshua Bengio. Generative adversarial nets. In Advances in neural information processingsystems, pages 2672–2680, 2014. 3

[11] Priya Goyal, Piotr Dollár, Ross Girshick, Pieter Noordhuis, Lukasz Wesolowski, Aapo Kyrola, AndrewTulloch, Yangqing Jia, and Kaiming He. Accurate, large minibatch sgd: Training imagenet in 1 hour. arXivpreprint arXiv:1706.02677, 2017. 3

[12] Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition.In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 770–778, 2016.6

[13] Briland Hitaj, Giuseppe Ateniese, and Fernando Pérez-Cruz. Deep models under the GAN: informationleakage from collaborative deep learning. CoRR, abs/1702.07464, 2017. 1, 2, 3

[14] Gary B. Huang, Manu Ramesh, Tamara Berg, and Erik Learned-Miller. Labeled faces in the wild: Adatabase for studying face recognition in unconstrained environments. Technical Report 07-49, Universityof Massachusetts, Amherst, October 2007. 5, 6

[15] Forrest N Iandola, Matthew W Moskewicz, Khalid Ashraf, and Kurt Keutzer. Firecaffe: near-linearacceleration of deep neural network training on compute clusters. In Proceedings of the IEEE Conferenceon Computer Vision and Pattern Recognition, pages 2592–2600, 2016. 1, 3

[16] Xianyan Jia, Shutao Song, Wei He, Yangzihao Wang, Haidong Rong, Feihu Zhou, Liqiang Xie, ZhenyuGuo, Yuanzhou Yang, Liwei Yu, et al. Highly scalable deep learning training system with mixed-precision:Training imagenet in four minutes. arXiv preprint arXiv:1807.11205, 2018. 3

[17] Arthur Jochems, Timo M Deist, Issam El Naqa, Marc Kessler, Chuck Mayo, Jackson Reeves, ShrutiJolly, Martha Matuszak, Randall Ten Haken, Johan van Soest, et al. Developing and validating a survivalprediction model for nsclc patients through distributed learning across 3 countries. International Journalof Radiation Oncology* Biology* Physics, 99(2):344–352, 2017. 1, 3

[18] Arthur Jochems, Timo M Deist, Johan Van Soest, Michael Eble, Paul Bulens, Philippe Coucke, Wim Dries,Philippe Lambin, and Andre Dekker. Distributed learning: developing a predictive model based on datafrom multiple hospitals without data leaving the hospital–a real life proof of concept. Radiotherapy andOncology, 121(3):459–467, 2016. 1, 3

[19] Hanjoo Kim, Jaehong Park, Jaehee Jang, and Sungroh Yoon. Deepspark: Spark-based deep learningsupporting asynchronous updates and caffe compatibility. arXiv preprint arXiv:1602.08191, 2016. 3

[20] Jakub Konecný, H. Brendan McMahan, Felix X. Yu, Peter Richtarik, Ananda Theertha Suresh, and DaveBacon. Federated learning: Strategies for improving communication efficiency. In NIPS Workshop onPrivate Multi-Party Machine Learning, 2016. 1, 3

[21] Alex Krizhevsky. Learning multiple layers of features from tiny images. Technical report, Citeseer, 2009.5, 6

[22] Yann LeCun. The mnist database of handwritten digits. http://yann. lecun. com/exdb/mnist/. 5, 6[23] Mu Li, David G Andersen, Jun Woo Park, Alexander J Smola, Amr Ahmed, Vanja Josifovski, James

Long, Eugene J Shekita, and Bor-Yiing Su. Scaling distributed machine learning with the parameter server.In 11th {USENIX} Symposium on Operating Systems Design and Implementation ({OSDI} 14), pages583–598, 2014. 1, 3

[24] Yujun Lin, Song Han, Huizi Mao, Yu Wang, and William J Dally. Deep gradient compression: Reducingthe communication bandwidth for distributed training. arXiv preprint arXiv:1712.01887, 2017. 8, 9

[25] Dong C Liu and Jorge Nocedal. On the limited memory bfgs method for large scale optimization.Mathematical programming, 45(1-3):503–528, 1989. 6

[26] H Brendan McMahan, Eider Moore, Daniel Ramage, Seth Hampson, et al. Communication-efficientlearning of deep networks from decentralized data. arXiv preprint arXiv:1602.05629, 2016. 1, 3

[27] Luca Melis, Congzheng Song, Emiliano De Cristofaro, and Vitaly Shmatikov. Exploiting unintendedfeature leakage in collaborative learning. CoRR, abs/1805.04049, 2018. 1, 2, 3, 5, 6

[28] Yuval Netzer, Tao Wang, Adam Coates, Alessandro Bissacco, Bo Wu, and Andrew Y Ng. Reading digitsin natural images with unsupervised feature learning. 2011. 5, 6

10

[29] Adam Paszke, Sam Gross, Soumith Chintala, Gregory Chanan, Edward Yang, Zachary DeVito, ZemingLin, Alban Desmaison, Luca Antiga, and Adam Lerer. Automatic differentiation in pytorch. In NIPS-W,2017. 3, 6

[30] Pitch Patarasuk and Xin Yuan. Bandwidth optimal all-reduce algorithms for clusters of workstations.Journal of Parallel and Distributed Computing, 69(2):117–124, 2009. 1, 2, 3

[31] Le Trieu Phong, Yoshinori Aono, Takuya Hayashi, Lihua Wang, and Shiho Moriai. Privacy-preservingdeep learning via additively homomorphic encryption. IEEE Transactions on Information Forensics andSecurity, 13(5):1333–1345, 2018. 9

[32] Benjamin Recht, Christopher Re, Stephen Wright, and Feng Niu. Hogwild: A lock-free approach toparallelizing stochastic gradient descent. In Advances in neural information processing systems, pages693–701, 2011. 3

[33] Alexander Sergeev and Mike Del Balso. Horovod: fast and easy distributed deep learning in tensorflow.arXiv preprint arXiv:1802.05799, 2018. 3

[34] Reza Shokri, Marco Stronati, Congzheng Song, and Vitaly Shmatikov. Membership inference attacksagainst machine learning models. In 2017 IEEE Symposium on Security and Privacy (SP), pages 3–18.IEEE, 2017. 2, 3

[35] Giuseppe Tagliavini, Stefan Mach, Davide Rossi, Andrea Marongiu, and Luca Benini. A transprecisionfloating-point platform for ultra-low power computing. CoRR, abs/1711.10374, 2017. 8

[36] Yusuke Tsuzuku, Hiroto Imachi, and Takuya Akiba. Variance-based gradient compression for efficientdistributed deep learning. arXiv preprint arXiv:1802.06058, 2018. 8, 9

11

deep leakage from gradients...dlg only requires the gradients and can reveal pixel-wise accurate...

Documents