safelearning: enable backdoor detectability in federated
TRANSCRIPT
SAFELearning: Enable Backdoor Detectability In Federated Learning WithSecure Aggregation
Zhuosheng ZhangStevens Institute of Technology
Jiarui LiStevens Institute of Technology
Shucheng YuStevens Institute of Technology
Christian [email protected]
AbstractFor model privacy, local model parameters in federated learn-ing shall be obfuscated before sent to the remote aggregator.This technique is referred to as secure aggregation. However,secure aggregation makes model poisoning attacks such back-dooring more convenient considering that existing anomalydetection methods mostly require access to plaintext localmodels. This paper proposes SAFELearning which supportsbackdoor detection for secure aggregation. We achieve thisthrough two new primitives - oblivious random grouping(ORG) and partial parameter disclosure (PPD). ORG parti-tions participants into one-time random subgroups with groupconfigurations oblivious to participants; PPD allows securepartial disclosure of aggregated subgroup models for anomalydetection without leaking individual model privacy. SAFE-Learning can significantly reduce backdoor model accuracywithout jeopardizing the main task accuracy under commonbackdoor strategies. Extensive experiments show SAFELearn-ing is robust against malicious and faulty participants, whilstbeing more efficient than the state-of-art secure aggregationprotocol in terms of both communication and computationcosts.
1 Introduction
Federated learning [23] becomes increasingly attractive inemerging applications [1, 2] with the proliferation of Internetof Things and edge computing. As compared to centralizedlearning (i.e., training models at the central server), federatedlearning allows participants (i.e., users) to locally train mod-els with their private data sets and only transmit the trainedmodel parameters (or gradients) to the remote server. The lat-ter aggregates local parameters to obtain a global model andreturns it to users for next iteration. However, recent researchhas discovered that disclosing local models also poses threatsto data privacy, either directly or under subtle attacks suchas reconstruction attacks and model inversion attacks [4]. Toprotect local models against disclosure, nobody except for the
participant shall know her own local model while the globalmodel will be revealed to all participants. This problem isknown as Secure Aggregation which can be generally real-ized using cryptographic primitives such as secure multipartycomputation (SMC) and homomorphic encryption (HE). De-spite of recent progresses, existing SMC or HE constructionsare still not fully ready for complex networks (e.g., in deeplearning) because of the high communication and/or comput-ing overhead. Differential privacy [11, 32] provides efficientalternative solutions to model privacy protection. However,it remains a challenge to maintain an appropriate trade-offbetween privacy and model quality in deep learning tasks.
Pairwise masking [17, 36] has recently caused attentionfor its efficiency in secure aggregation. Specifically, assumeA and B have respective parameters xa and xb and a sharedpairwise mask sa,b. They simply hide their parameters by up-loading ya = xa + sa,b and yb = xb− sa,b (with an appropriatemodulus) respectively. The shared mask will be cancelled dur-ing aggregation without distorting the parameters. While theshared mask can be conveniently generated using well-knownkey exchange protocols and pseudo random generators, themain problem is to deal with dropout users who become of-fline in the middle of the process and make the shared masknot cancellable. To address this problem, Bonawitz et. al. [9]proposed a protocol that allows online users to recover offlineusers’ secrets through secret sharing. Without heavy crypto-graphic primitives, [9] supports secure (linear) aggregationfor federated learning without accuracy loss as long as thenumber of malicious users is less than a threshold number t.
0 2000 4000 6000 8000 10000 12000 14000Local Parameter Scale Factor
0.0
0.2
0.4
0.6
0.8
1.0
Accu
racy
Main taskBackdoor
Figure 1: Backdoor/main model accuracy of [9] with one attacker.
One outstanding problem with secure aggregation is thatmodel poisoning attacks are made easier. This is because
1
arX
iv:2
102.
0240
2v2
[cs
.CR
] 3
Sep
202
1
local models are no longer revealed to the aggregator, whichis required in existing Byzantine attack detection techniques[15]. By uploading scaled erroneous parameters, attackerscan launch model poisoning attacks through label-flipping [7]or model backdoor [5], with the aim to manipulate the globalmodel at the attacker’s will. As shown in Fig. 1, a backdoorcan be conveniently inserted to the global model in [9] evenif one single attacker is presented. To our best knowledge, itstill remains a challenge to simultaneously support backdoordetection and secure aggregation in federated learning.
In this paper we design a new protocol, namely SAFE-Learning, to address backdoor attacks in secure aggregation.We achieve this with two new primitives - oblivious randomgrouping (ORG) and partial parameter disclosure (PPD).ORG partitions users into one-time random subgroups ateach iteration with both group membership information andsubgroup configurations (including the number of subgroups)oblivious to users (membership information is also obliviousto the aggregation server). Neither users nor the aggregationserver can predict or determine which user is assigned towhich subgroup before local parameters are uploaded. Thisproperty forces attackers to work independently even if theyare willing to collude. By making subgroup configurationsoblivious, we further thwart opportunistic attackers who inde-pendently manipulates local parameters based on statisticalestimation of the distribution of attackers in subgroups. PPDsupports secure partial disclosure of aggregated models ofsubgroups by limiting the privacy leakage to the level thatthe global model can leak. With ORG and PPD, the aggrega-tion server is able to randomly evaluate subgroup models foranomaly detection without additional privacy leakage.
As compared to Bonawitz et. al., the computational com-plexity of SAFELearning is reduced from O(N2 +mN) (atuser side) and O(mN2) (at server side) to O(N +n2 +m) andO(mN + nN), respectively, where N is the total number ofusers, n (� N) the number of users in subgroups and m thesize of the model. This is attributed to the hierarchical sub-group design in ORG. SAFELearning is provably secureunder the simulation-based model. We conducted extensiveexperiments with ResNet-18 network over CIFAR-10 datasetunder well-known backdoor strategies. Experimental resultsshow that SAFELearning is able to completely defeat back-door attacks (i.e., reducing backdoor accuracy to 0) in mostcases when the percentage of attackers is relatively low. Mean-while, SAFELearning exhibits similar convergence rate ascompared to vanilla federated learning. Table 1 comparesSAFELearning with existing representative secure federatedlearning techniques.The main contributions of this paper are as follows: 1) wedesign a new secure aggregation protocol that simultaneouslysupports backdoor attack detection and model privacy; to ourbest knowledge, this work is among the first that can detectmodel-poisoning attacks on encrypted model parameters forfederated learning; 2) the proposed scheme significantly im-proves system scalability in both computational and commu-
Table 1: Related secure federated learning techniques
MethodPrivacy
PreservationBackdoordetection
Computationcomplexity
SecureFL [9] Yes No O(mN2)
Detection [8] [15] No Yes O(N2)
SAFELearning Yes Yes O(mN +nN)
nication complexities as compared to state-of-the-art secureaggregation techniques; 3) SAFELearning is provably secureunder the simulation-based model.
This rest of the paper is organized as follows. Section 2presents models and technical preliminaries. An overview toour design is described in Section 3. Section 4 elaborates ourORG primitive and secure aggregation protocol. Section 5 ex-plains the PPD design and our backdoor detection mechanism.Section 7 presents experimental evaluation of our backdoordetection algorithm. Section 8 review related work. We con-clude the paper in Section 9.
2 Models and Preliminaries
2.1 System Model
We assume two types of parties in a federated learning system:an aggregation server S and a set of N participating users U .The server holds a global model Xi of size m and each useru ∈U possesses a private training data set. Users train theglobal model shared by the server with their private trainingdata at each iteration and upload the local model parametersto the server. The server aggregates local parameters andcompute ∑u∈U xu, where xu (also of size m) is the local modelparameter trained by u using Xi and his local data. The serverreturns the latest global model to each user at the end of eachiteration. The server communicates with each user througha secure (private and authenticated) channel. A trusted thirdparty authenticates each user and generates a private-publickey pair < sSK
u ,sPKu > for her before the protocol execution.
Each user u is assigned a unique “logical identity" Idu in a fullorder. Users may drop out at any phase of the protocol but atleast t users are online for any randomly selected subgroup ofn users at any time. In the training phase, we assume the serveruses the baseline federated learning algorithm FedSGD [26]with the following update rule, where η is the learning rate:Xi+1 = Xi +
η
N ∑xu∈U
(xu−Xi).
2.2 Attack Model
We consider two types of attackers who have distinct objec-tives: type I attackers are interested in learning the value oflocal parameters to comprise model privacy; and type II at-tackers are motivated to poison the global model to generate
2
a model with semantic backdoors. The global model in typeII attack shall exhibit a good main task accuracy (or “mainaccuracy" for short) but also behave in a way at attacker’s willon attacker-chosen backdoor inputs.
Type I attackers: Both the server and malicious users areconsidered potential type I attackers but the server is assumedhonest but curious. The server may collude with malicioususers to compromise model privacy of benign users. However,the ratio of users it can collude does not exceed r
N for anyrandomly chosen subgroup of users. We assume the server isagainst type II attackers since it may benefit from an accurateand robust global model.
Type II attackers: For type II attackers, we follow similarassumptions of [5], i.e., attackers have full control over oneor several users, including their local training data, models,and training settings such as learning rate, batch size and thenumber of epochs. However, attackers are cannot influencethe behavior of benign users. The number of type II attackers(malicious or compromised users) are assumed to be muchsmaller than benign users. Further more, we assume attackerscan fully cooperate with each other, including sharing theirsecret keys, whenever necessary. To be specific, we define theobjective of type II attackers Ua in each iteration is to replacethe aggregated global model Xi+1 with a target model Xtarget1 as shown in (1). Uh (Ua) refers to the set of benign users(attackers) in this iteration.
Xi+1 = Xi +η
N ∑xu∈Uh
(xu−Xi)+η
N ∑xa∈Ua
(xa−Xi)≈ Xtarget
(1)
2.3 Attacker StrategiesTo yield a target model Xtarget , each attacker shall constructlocal parameters xa approximately base on the following:
xa−Xi =N
η|Ua|(Xtarget −Xi)−
1|Ua| ∑
xu∈Uh
(xu−Xi)
≈ Nη|Ua|
(Xtarget −Xi)
(2)
To make the attack more effective and stealthy, the follow-ing possible strategies might be adopted by attackers :
Sybil attacks: In order to reduce the scaling factor γ =N
η|Ua| and make the attack stealthy, attackers tend to deploy asmany adversary participants as possible, e.g., by Sybil attacks[14] to increase |Ua|. In this work, we assume the trustedparty authenticates each user when issuing public/private keypairs to thwart sybil attacks.
Adaptive Attacks: Strategic attackers can launch adaptiveattacks [5, 6] by including the distance Distance(Xtarget ,Xi)between Xtarget and Xi (either geometric distance or cosine dis-tance of gradients) in the loss function while training Xtarget .
1Xtarget can be a transitional model in continuous attacks.
The purpose is to reduce the term Xtarget −Xi in (2) to makeattack more imperceptible. However, this term cannot be arbi-trarily optimized if the attackers’ objective is different fromthe main task of the model.
0 25 50 75 100 125 150 175
104
106learning rate=10e-2learning rate=10e-4
Figure 2: Continuous attacks - scale factor vs. number of iterations
Continuous Attacks: To make the attack stealthier, attack-ers may instead choose to continuously perform the attackthrough multiple iterations, expecting a relatively smallerscale factor at each iteration. However, our preliminary exper-iments in Fig. 2 discovers that a minimum parameter scalefactor is required for successful backdoor attacks. The scalefactor is proportional to the inverse of learning rate whichis controlled by the server. Previous research [35] [5] alsodiscovered that rationale attackers would launch the attacktoward the convergence time. This is because if launched inearly stages the backdoor tends to be "forgotten" and the maintask will be jeopardized due to the disturbance of the attack.
Therefore, in this paper we assume attackers need to scaleup their model to a certain minimum level even with continu-ous attacks. We also assume the attack will not be launchedat the very beginning of training but rather in the middle ofit to reduce the disturbance on the main task. We considerall the above attack strategies and their combinations exceptfor unlimited Sybil attacks. Following existing research [9],we assume the attackers only count for a small portion of theentire population of users (i.e., |Ua| � N).
2.4 Secure Aggregation with Pairwise Mask-ing
Recently, pairwise additive masking [9, 17, 36] has been uti-lized as an efficient cryptographic primitive for secure aggre-gation in federated learning even for complex deep networks.As this paper utilizes pairwise masking for secure aggrega-tion, we provide overview of a recent secure aggregationscheme [9] as follows.
Let xu denote the an m-dimensional vector of parametersthat user u ∈U generates locally, where U is the set of allusers. Assume a total order of users and each user u is as-signed a private-public key pair (sSK
u ,sPKu ). Each pair of users
(u,v), u < v, can agree on a random common seed su,v usingDiffie-Hellman key agreement [13]. With the seed, a com-mon mask vector PRG(su,v) can be computed by u and vusing a pseudo-random generator (PRG) (e.g., a hash func-tion). When u obfuscates her parameter vector xu by addingthe mask vector and v subtracting it, the mask vector will becanceled when server aggregates the obfuscated parameter
3
vectors without reveal their actual values. Specifically, eachuser u obfuscates her parameter xu as following:
yu = xu + ∑∀v∈U :u<v
PRG(su,v)
− ∑∀v∈U :u>v
PRG(sv,u) (mod R)
and sends yu to the server. Then the server computes:
z = ∑u∈U
(xu + ∑
v∈U :u<vPRG(su,v)− ∑
v∈U :u>vPRG(sv,u)
)= ∑
u∈Uxu (mod R)
To address user dropouts, each user u creates N sharesof her secret sSK
u using Shamir’s t-out-of-N secret sharingscheme and sends the shares to the rest of users. Additionally,each user u generates another random seed bu which is mainlyto prevent the aggregation server from learning her parametervectors in case she is delayed but her secret has been recoveredby other users before she becomes online and sends out yu.Random shares of bu are also generated and sent to otherusers. Each user u obfuscates the parameter vector xu using amask PRG(bu) in addition to the pairwise mask vector:
yu = xu +PRG(bu)+ ∑∀v∈U :u<v
PRG(su,v)
− ∑∀v∈U :u>v
PRG(sv,u) (mod R)
In the unmask round, for each dropped user v, online usersreveal the shares of sSK
v ; for each online users u, other onlineusers reveal the shares of bu. Then the server will be able tocompute PRG(sv,u) and PRG(bu) for any online user u andcancel it out from the sum z to get the aggregate model ofonline users. Note that an honest user u never reveals eithershares of s j,v or bv for any user v before the unmask round.
The scheme saliently protects confidentiality of local pa-rameters efficiently while taking into account user dropoutsin practical distributed systems. However, the solution alsomakes it convenient for model poisoning attacks (e.g., back-door attack). As pointed out in a recent work by Bagdasaryanet. al. [5], even a single malicious user is able to manipulatethe global model through model replacement attack. This ispossible because secure aggregation fully encrypts the users’local model, which allows the attacker to submit any erro-neous parameters. As the aggregation server does not neces-sarily have access to validation datasets, such attack is difficultto detect by simple model validation.
3 Overview of SAFELearning Design
Achieving both model privacy and backdoor detectabilityseems challenging because of the conflicting natures of thetwo objectives. To address this issue, in SAFELearning we
organize users into subgroups with a hierarchical k-ary treestructure as shown in Fig. 3. At the leaves are equal-sizedsubgroups of n users. At the aggregation server, the models ofeach subgroups are first aggregated; the aggregated model ofthe subgroups are further aggregated at the next level of sub-groups; the process repeats recursively toward the root of thetree. It is trivial to show that the aggregated global model re-main the same as exiting federated learning algorithms. Withthe tree structure, users in the same subgroup pairwise “mask"each other during the secure aggregation process. To protectprivacy of the aggregated model of each subgroup, a pairwisemask is also generated for each subgroup at internal layersof the tree as shown in dashed line in Fig. 3. Similarly, usersecrets (i.e., sSK
u and bu for user u as discussed in Section 2.4)can be securely shared within subgroups. Intuitively, secureaggregation with the tree structure provides similar level ofprotection to model privacy of users as in [9], and the userdropouts can be handled similarly as well. As secret sharingis within subgroups, we directly enjoy the benefit of reducedcomplexity, i.e., from O(N2) in [9] to O(n2), because of thehierarchical group design.
However, strategic type I attackers can comprise localmodel privacy by deploying an overwhelming number of mali-cious users in a target subgroup. This is possible when n� Nand |Ua| 6� n. To prevent such attacks, we shall not alloweither the server or any user to determine which users belongto which subgroups. Specifically, the assignment of users tosubgroups shall be randomized so that nobody can assignherself or others to a target subgroup with a non-negligibleprobability more than random assignment.
The randomized subgroup assignment also provides theopportunity for detecting type II attackers, whose purposeis to manipulate the aggregated global model, e.g., to insertbackdoors. Specifically, model poisoning attackers need toamplify their local parameters dramatically in order to influ-ence the global model as discussed in Section 2.3, no matterin “one-shot" last-round attacks or continuous attacks. If eachattacker were to work independently, the magnitude of aggre-gated models at the subgroups will differ significantly fromeach other unless each subgroup has exactly the same num-ber of attackers, the chance of which however is very lowdue to the randomness of the tree-based subgroup assignmenteven if the number of attackers equals the number of sub-groups. The aggregation server can make the chance evenlower by randomly changing subgroup configurations overiterations. Considering collaborative attackers, however, suchrandomness alone is not enough for backdoor attack detection(or model poisoning detection in general). This is becausecollaborative attackers can intentionally adjust the scales oftheir local parameters to make the distribution of aggregatedsubgroup models uniform unless subgroups (at leaf layer)outnumbers attackers. To defeat such collaboration, we needto make the attackers oblivious to each other’s subgroup mem-bership information. This means that the attackers shall not
4
server
Layer 2
Layer 1
Layer
1 2 n 1 2 n 1 2 n1 2 n 1 2 n 1 2 n1 2 n 1 2 n 1 2 n
Figure 3: The random tree structure (blue and green dotted lines indicate subgroups at levels 2 and 3 respectively).
know whether or not a given user/attacker belongs to whichsubgroup. We call such subgroup assignment as obliviousrandom grouping (ORG).
With the tree-based ORG, we can detect model poison-ing attacks by evaluating aggregated models of the subgroups(i.e., those at the leaf layer of the tree). Specifically, subgroupswith more attackers will have much higher magnitude in theiraggregated models because of the scale-up of attackers’ pa-rameters unless each subgroup has exact number of attackers,the chance of which is extremely low in large-scale systems asshown in Section 4.3. However, directly revealing aggregatedmodels of subgroups may also lead to some privacy leakagedepending on the number of users in the subgroup. To addressthis issue, SAFELearning only reveals partial higher bits ofaggregated subgroup models to the extend that the privacyleakage is no more than what is disclosed by the global model(which is public to all users anyway). Such partial parameterdisclosure (PPD) allows us to compare aggregated models ofsubgroups and even to conduct some statistical analysis withmodel privacy preserved.
The high-level workflow of our protocol is as following:(a) first, the users and server work together to generate therandom tree with our tree generation sub-protocol; at the endof this step, the tree structure and the full orders of secretsharing and pair-wise masking are determined; (b) next, eachuser shares his secret keys sSK
u and bu to the users in thesame subgroup, obliviously masks his input xu according tothe tree structure (without knowing the subgroup member-ship information), and sends the encrypted input to the server.After having collected the input and secret shares from theusers, (c) the server compares the partial information of theaggregated model from each subgroup to detect abnormalsubgroup(s). It computes the global model and returns it tousers but without using parameters of abnormal subgroups.ORG is implemented through steps (a) and (b), and PPD isrealized in steps (b) and (c). Next two sections elaborate ourdesign of tree-based secure aggregation and poisoning attackdetection.
4 Tree-Based ORG and Secure Aggregation
4.1 Tree Structure DefinitionWe define two independent random tree structures Tshare andTmasking, both of which share the same structure of Fig. 3 butwith independent subgroup membership assignments. Tshareis for secret sharing of secret keys sSK
u and bu, and Tmasking isfor pairwise masking of model parameters. Decoupling thetwo trees is because of the different security requirements ofsecret sharing and pairwise masking. For secret sharing, treeTshare divides users into small subgroups (rounded rectangleshows in Fig. 3). The secret sharing will perform betweenusers inside the same subgroup at the leaf layer of the tree.
Secure aggregation is performed as in Section 2.4 but us-ing pairwise masks within subgroups. Each user applies twotypes of masking to her parameters - intra-group maskingand inter-group masking. Intra-group masking is to protectindividual local models using pairwise masks of users withinthe same subgroup at the leaf level of the tree. The aggregatedmodel obtained at this level is called subgroup aggregatedmodel. Inter-group masking is to protect the subgroup aggre-gated models, and the pairwise masks are generated betweenpeers at higher non-leaf layer subgroups. As shown in Fig. 3,we logically form groups (represented as Gi j, i is the layernumber and j means this group is the j-th child node of itsparent) over k subgroups at each non-leaf layer of the tree.This process repeats recursively toward the root. Specifically,for each user u a pairwise masking peer v should comply withfollowing rules:
(1) Ancestors of u and v immediately under their leastcommon ancestor (LCA) shall be within κ immediate neigh-borhood based on the ancestors’ total oder at that layer, whereκ is a system parameter.
(2) The positions of u and v shall be the same by their totalorders in their respectively sub-trees, so are the positions oftheir ancestors below the immediate children of their LCA.
(3) Only two peers are needed at each layer for each node.Based on these rules, when κ = 1 user 2 in G11 of G21 in
Fig. 3 has the following peers: users 1 and 3 in the samesubgroup; user 2 in G12 of G21 and user 2 in G1k of G21(because G11 is the neighbor (mod k) of G12 and G1k at layer1); user 2 in G11 of G22 and user 2 in G11 of G2k (becauseG21 is the neighbor (mod k) of G22 and G2k at layer 2); so on
5
and so forth. This is illustrated by the dotted line in Fig. 3.Therefore, our inter-group masking have following properties:first, the number of pairwise masking operations for each useris 2 log N
n , twice of the tree height; second, those pairwisemasks cannot be cancelled until the server aggregates all thesubgroups’ aggregated models at that layer of the tree. LetGs
u be the intra-group masking peers of user u (i.e., pairwisepeers in the same subgroups as u) and Gp
u inter-group maskingpeers. The masking equation for user u can be written as
yu = xu +PRG(bu)+ ∑∀v∈{Gs
u,Gpu}:u<v
PRG(su,v)
− ∑∀v∈{Gs
u,Gpu}:u>v
PRG(sv,u) (mod R)(3)
The server is able to recover a dropped user u’s pairwisemasks of any type by secure recovery of his private key sSK
uif at least t honest users in the dropped user’s secret sharingsubgroup survived.
4.2 Tree-Based Random Subgroup Genera-tion
For secure ORG, the first requirement is the randomness ofthe subgroup membership assignment. As users are groupedin the total order by their identities, the randomness can beassured if user identities are randomly generated, i.e., they arerandom and not solely determined by either the user herselfor the server. Specifically, user identity Idu for user u is gener-ated as following2: Idu = HASH(Rs||cPK
u ||Ru), where cPKu (or
sPKu if it were the pairwise mask tree Tmask) is the public key
used in Diffie-Hellman key agreement and is used one-timefor each iteration. Random numbers Rs and Ru are generatedby the server and user u respectively. Because of the random-ness of hash function, Idu is randomly distributed and notpredictable to both users and the server.
For random subgroup assignment, however, the order ofthe disclosure of the tree structure T (generated by the server)and that of Idu (jointly produced by the server and user u)is important. In particular, if Idu is disclosed before T , theserver might be able to intentionally group certain users in asubgroup by adjusting the tree structure. On the other hand,if T is disclosed before Idu, malicious users could attemptto group themselves together by manipulating their identi-ties (e.g., via finding special hash results). Both could leadto model privacy disclosure of victim users. To defeat suchpotential attacks, we design a commitment protocol, as shownin Fig. 4, with which T and Idu are committed before disclo-sure. Specifically, the server first broadcasts its commitmentof a random number Rs. Users then send their public key cPK
u(or sPK
u if it were pairwise mask tree Tmask) and commitmentof Ru to the server. After collecting enough users, the server
2It works similarly for Tshare and Tmask. In our next description, we takeTshare as example.
will decide the tree structure (degree and layers) T base onthe number of users N. At this point, Idu has been uniquelydetermined but yet disclosed; neither users nor the server cancompute or predict it. And the tree structure is determined bythe server independently to user identities. The server thenbroadcasts the commitment of T , Rs and a list of commit-ments to Ru’s. On receiving the broadcast message from theserver, user discloses Ru to the server, allowing the server tocompute the user’s Idu and make subgroup assignment. Userscan verify the correctness of the protocol by requesting theserver to broadcast T and the list of cPK
u (or list of sPKu ) and Ru
after local parameters have been sent out.
，
Users Server
Verify
Figure 4: Tree structure generation protocol.
The protocol has considered misbehaving server and users.However, there is still risks when the server colludes withmalicious users. For example, they can pre-compute Idu byexhaustively testing different Rs and Ru to obtain special iden-tities, e.g., those with leading zero(s). As benign users are lesslikely to have such special identities due to randomness of thehash function, the malicious users will be assigned to the samesubgroup and dominate that group. To defeat such attacks,we generate the final identity of user u as HASH( ∑
∀v∈G:v 6=uIdv)
instead of using Idu directly, where ∑ means XOR. By this,the randomness is determined by all but the user himself. Withrandom and trustworthy full order identities of all users andan independently generated tree structure, we can achieverandom subgroup assignment as shown in Fig. 3.
4.3 Oblivious Secure AggregationAs discussed, users shall be oblivious to subgroup member-ship information for secure ORG. Otherwise, malicious usersare able to coordinate and manipulate the distribution of thesubgroup aggregated model parameters to bypass anomalydetection. In particular, if attackers (malicious users) outnum-bers the subgroups (at the leaf layer), they can coordinate andstrategically adjust local parameters to make the distributionof subgroup aggregated models uniform. If the subgroupsoutnumbers attackers, however, the distribution is doomedimbalance unless attackers give up the attack. Without coordi-nation, the chance that each subgroup contains exact the samenumber of attackers is very low. For example, if there are xattackers and x subgroups, the probability that each subgroupshave exactly one attacker is x!
xx .
6
However, if users are oblivious to subgroup membership,they are not able to identify their peers and generate pairwisemasks to encrypt local parameters. To solve this problem, welet the server (who is against model poisoning attacks for itsown benefits) directly send each user the list of public keysof all the users who are her pairwise mask peers. However,directly sending original public keys may allow malicioususers to recognize each other and know their group mem-bership information. To address this problem, we design arandomized D-H key exchange protocol wherein the serverrandomizes each user’s public key before sending it out. Withthis randomized public key, two malicious users are not ableto tell whether or not they belongs to the same subgroup un-less they are pairwise peers. Specifically, our construction isas follows.
Randomized D-H key exchange. Assume the server is tocoordinate the exchange of public keys between users u and v,with their respective public keys sPK
u = gsSKu and sPK
v = gsSKv .
To prevent them from recognizing each other’s public key,the server “randomizes" their public keys before sending out.Specifically, it first produces a random number ru,v, and thensends a randomized public key s(PK,v)
u = (sPKu )ru,v to user v
and s(PK,u)v = (sPK
v )ru,v to user u. After key exchange like D-H,the shared key su,v will become following form:
su,v = (s(PK,v)u )sSK
v = (s(PK,u)v )sSK
u = gsSKu ∗sSK
v ∗ru,v
Please note that pairwise peers are still able to verify thatthey are in the same subgroup by comparing the shared keythey computed. However, the purpose of our randomized D-Hkey exchange is to thwart users who are in the same subgroupbut not peers from knowing the fact that they are in the samesubgroup. This is achieved because of the unique randomnumber ru,v for each pair of peers. Without the randomization,however, two attackers will easily know that they in the samegroup if they receive common public keys of benign user(s).
(a) Complete graph topology (b) Circular topology
Figure 5: Graph representation of users inside a subgroup. Rednodes represent attackers. Edges denote pairwise masking relation-ship.
Circular topology for pairwise masking relationshipConsider users inside a subgroup as vertices of an undirectedgraph, the edges of which represent pairwise masking peerrelationship as shown Fig. 5. If users choose all the otherusers in the subgroup as peers, the graph is complete. In thiscase attackers will know each other’s membership informa-tion because each pair of peers share the same seed su,v. To
avoid such situation, we use a circular graph as shown in Fig.5 (b), in which each user u only pairs with 2κ users - her κ
immediate previous and next neighbors based on the totalorder Idu. For successful attack, attackers need to know themembership information of all of them. This is possible onlywhen the attackers inside the same subgroup form a chain.Assume there are a total of x attackers in a subgroup, thechance is at most kκ
nk−1 , where k is the number of attackers insame subgroup and n is the size of the subgroup.
5 Backdoor Attack Detection
In machine learning, one approach to detect model poisoningor Byzantine attacks is by statistically analyzing the magni-tude of the model parameter vectors [8, 31]. As discussedin Section 2.3, attackers need to amplify model parametersby a factor of γ = N
η|Ua| on average to successfully launchmodel poisoning attacks. With random subgroup assignment(Section 4.2) and oblivious secure aggregation (Section 4.3),users are partitioned into random subgroups at each itera-tion and perform pairwise masking without knowing others’group information. Moreover, the random tree structure (in-cluding the subgroup configuration) is not revealed beforelocal parameters of all users have been uploaded (Section 4.2).Attackers have to work independently and it is difficult for anyof them to control or even predict the distribution of subgroupaggregated models. As a result, the subgroup aggregated pa-rameters will be very sensitive to even slight differences inthe number of attackers presented. This provides the oppor-tunity to detect backdoor attacks by comparing the subgroupaggregated models.
In our secure aggregation scheme, local parameters aremasked by both intra-group masks and inter-group masks.After aggregation at leaf-layer subgroups, the subgroup ag-gregated models are only protected by inter-group masks. Bychanging inter-group masks from full-bit masks to partialmasks, i.e., by revealing few higher bits, we can observe par-tial information of subgroup aggregated models and performanomaly detection. Because attackers need to significant scaleparameters, non-zero bits may present in higher bits for at-tackers’ subgroups. However, disclosure of these higher bitsis acceptable only if it does not lead to extra privacy leakageas compared to what the global model discloses. To this endwe first derive the number of bits that can be disclosed bythe partial parameter disclosure (PPD) mechanism and thenpresent our backdoor detection algorithm.
5.1 Privacy-Preserving PPDBit format of model parameters and masking: Assumeeach element of xu is in range [0,RU ] and can be stored infix-point format3, and the high bit segment H(xu) in range
3Previous research [19] discovered that using 14 bits fix point data onlyhave 0.05% accuracy decreasing compare to 32 bits float point data in MNIST
7
[RH ,RU ] can be disclosed. The extracted high bits of the ag-gregated model of subgroup G1i is {∑xu∈G1i H(xu)}i∈N,i≤N
n.
G1i is i-th subgroup at the leaf layer (i.e., layer 1 as shown inFig. 3). To support higher bits disclosure, we can adjust thepairwise masking equation for user u as follows:
yu = xu +PRG(bu)+ ∑∀v∈Gs
u:u<vPRG(su,v)−
∑∀v∈Gs
u:u>vPRG(sv,u)+ ∑
∀v∈Gpu :u<v
(ΛRH ∧PRG(su,v))−
∑∀v∈Gp
u :u>v
(ΛRH ∧PRG(sv,u)) (mod R) (4)
where ΛRH is a vector of binary masks of length m andGp is the set of inter-group peers. The bits at positions in[log2RH , log2RU ] of ΛRH are set as 0’s, and the rest bits are 1.In this way, higher bits starting from log2RH are not masked.
To estimate the privacy disclosure by revealing the higherbits of the subgroup aggregated models, we analyze the poste-rior probability P(|Xi− y|< ε|Y ), where Xi ∈ Rm is the localparameter vector, y is the aggregation result and Y is its range.We compare the probability bound of aggregating all N usersbut revealing all the bits (i.e., the disclosure of global model)with the bound of aggregating n users but revealing only bitsin range [RH ,RU ] (i.e., disclosure of higher log2RU − log2RHbits of subgroup aggregated models). We prove that when
RH = 2(1−√
n−1n )ε, the expected bounds will be the same
in the two situations.
Theorem 1. Let {Xi}i∈[1,N] be N samples from some arbitrarydistribution with mean vector E(X) and variance σ
2N . The
probability bound is:
P(||Xi− y|| ≥ ε|y = E(X))≤ σ2N
ε2 ,ε > 0 (5)
Proof. This can be proved by Chebyshev Inequality directly.
Theorem 1 shows that if the server aggregates at root node,which is equivalent to Y = y = E(X), the difference betweenlocal vector Xi and aggregated result y is limited by the vari-ance of X .
Theorem 2. Let {X ′i }i∈[1,n] be n samples randomly se-lected from {Xi}i∈[1,N] in Theorem 1 and Y = y ∈ [E(X ′)−RH/2,E(X ′) + RH/2], where E(X ′) is the mean vector of
{X ′i }i∈[1,n]. If RH = 2(1−√
n−1n )ε, the expected bound of
P(|Xi− y|< σ|Y ) is σ2N
ε2 .
Proof. Please see Appendix A for detailed proof.
and CIFAR-10 dataset.
Theorem 2 indicates that information leakage can be re-duced by increasing the subgroup size n or decreasing thenumber of bits revealed (i.e., increasing RH ). Therefore, thethe server can adjust parameters n and RH to minimize privacy
discourse risk. In particular, when RH = 2(1−√
n−1n )ε, the
privacy leaked by disclosing the higher log2RU − log2RH bitsis the same as what is disclosed by the global model.
5.2 Backdoor Attack DetectionWith the partially revealed subgroup aggregated parameters,we design a new backdoor attack detection algorithm (asshown in Algorithm 1) to detect and withstand backdoorattacks. At each iteration t, we first compute the Euclideandistance di between the global model H(Xt) and the partiallyrevealed aggregated model H(xi) for each subgroup i. Thenthe server computes the standard deviation Std(D)t of {di} foriteration t. An adaptive threshold ξt =
ρ
T ∑Tj=1 Std(D)t− j is
also computed, which is the product of the average of standarddeviations in last T iterations and a threshold expansion factorρ. If the standard deviation Std(D)t is above the threshold ξt ,the server replaces the subgroup with the largest distance tothe global model H(Xt) with H(Xt) (i.e., the partially revealedhigher bits), and recomputes the standard deviation of theupdated {di}. This repeats until the standard deviation of {di}is below the threshold ξt .
To better understand how our method works, we differ-entiate three attack scenarios: all benign subgroups, mixedsubgroups and all malicious subgroups. When no subgroupis malicious, our method can avoid false positive detectionbecause our adaptive threshold is proportional to the aver-age of standard deviation of {D} in the last few rounds, andthe threshold expansion factor ρ will relax the threshold toaccept normal deviations introduced by benign users. Whenbenign subgroups and malicious subgroups are both presented,which usually happens when the number of attackers is lessthan or close to the number of subgroups, our method willreplace the aggregated models of malicious subgroups, whichusually have larger deviations from the global model as dis-cussed in Section 2.2, with the current global model. Whenall subgroups are malicious, which usually happens when theattackers outnumber the subgroups, the subgroups with moreattackers will be replaced with the global model first. Theremaining malicious subgroups will have large deviationsfrom the global model and result in a large standard deviationStd(D)t and will be subsequently replaced, which means thisiteration of training is abandoned.
One situation is when the attacker continuously manipu-lates the standard deviation for multiple rounds to neutralizethe adaptive threshold by injecting continuous attacks to grad-ually increase the standard deviation of {di}. However, asdiscussed in Section 2.3, there exist minimal scale factorseven for continuous attacks to be effective. Such scaled fac-tors are proportional to the inverse of the learning rate, a
8
0 50 100 150 200 250 300Iterations
10
20
30
40
50St
anda
rd d
ivia
tion
1.00
1.25
1.50
1.75
2.00
2.25
Mai
n ta
sk lo
ss
Standard diviationMain task loss
Figure 6: SAFELearing when no attacker exists - main task loss &standard deviation vs. iterations
hyperparameter controlled by the server. Moreover, withoutmalicious attackers, the standard deviation of {di} tends todecrease over iterations (even for non-i.i.d data), following thesimilar pattern of the global model’s loss function value. Thisis because when the global model is converging, the changerate of the global model and contributions from benign users’local models will decrease. This can be validated by our pre-liminary experiments as shown in Fig. 6. Therefore, whencontinuous attackers first join the training, we can expect alarge disparity between the standard deviation of current it-eration and previous iterations, which provides opportunitiesfor attack detection. Notes that if the attack starts from thevery beginning of training, the main task will be jeopardized.Detail evaluation of such attackers will be present in Section7.
Algorithm 1: Suspicious subgroup detection
Data: XL is the set of partially revealed aggregatedparameter vectors of the subgroup set GL atlayer L; Xt is global model of round t; Eucl(,)computes the Euclidean distance; Std(·)computes the standard deviation; ξt is adaptivethreshold of round t; ρ is the thresholdexpansion factor; M =∅; D =∅
Result: Attacker-inclusion set Mfor each vector xi ∈ XL do
Compute the distance to global modeldi = Eucl(H(xi),H(Xt))
D = D∪{di}endξt =
ρ
T ∑Tj=1 Std(D)t− j
while Std(D)> ξt doReplace the farthest subgroup with global
model H(xi) = H(Xt)D = (D\{di|di = Max(D)})∪{0}M = M∪{i}
end
5.3 Numerical Complexity
We present overall cost of our protocol in Table 2. Detailedcomplexity analysis and simulation results are elaborated inAppendix D and E, respectively.
Table 2: Overall computational and communication costs
Tree Based Secure Aggregation ProtocolUser Server
computation O(N +n2 +m) O(mN +nN)communication O(N +m) O(N2 +mN)
6 SECURITY ANALYSIS
We show the security of our protocol with the following theo-rems, where t ≥ |Ua| denotes the threshold number of attack-ers, C ⊆U ∪{S} an arbitrary subset of parties, x∗ the input ofa party or a set of parties ∗, U the set of all users, S the serverand k the security parameter.
Theorem 3. (Local Model Privacy under Type I Attackers)There exists a PPT simulator SIM such that for all t,U,xUand C ⊆ U ∪ {S}, where |C\{S}| ≤ t, the output of SIMis computationally indistinguishable from the output ofRealU,t,k
C (xU,U)
RealU,t,kC (xU,U)≈ SIMU,t,k
C (xC,z,U)
where
z =
{∑u∈U\C xu if |U| ≥ t⊥ o.w.
Proof. Detailed proof is presented in Appendix B.
Theorem 4. (Random Tree Structure Secrecy) There exists aPPT simulator SIM such that for all t,U,xU and C⊆U ∪{S},where |C\{S}| ≤ t, the output of SIM is indistinguishablefrom the output of real protocol:
RealU,t,kC (Ru,RS,CPK
u )≈ SIMU,t,kC (Ru,RS,CPK
u )
Proof. We prove this by a standard hybrid argument. Thedetailed proof is in Appendix B.
Theorem 5. (Indistinguishability of Type II attackers in thesame subgroup) For any type II attacker A in a certain sub-group, if there is another type II attacker B in the same sub-group but is not peered with A for pairwise masking, the jointview of A and B ViewA ,B is indistinguishable from ViewA ,the view of A:
ViewA ≈ViewA ,B
Proof. The detailed proof is in Appendix C.
7 Backdoor Attack Detection Evaluation
In this section, we evaluate our protocol against Type II At-tacker by testing our protocol under state-of-the-art semanticbackdoor attack [5] during the training of ResNet-18 net-work [20] on CIFAR-10 dataset. The examples of backdoordata are shown in Fig. 8 and the effectiveness of targeted
9
0 50 100 150 200 250 3000.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8Main accuracy
0 50 100 150 200 250 3000.0
0.2
0.4
0.6
0.8
1.0Backdoor accuracy
0 50 100 150 200 250 3000.0
0.2
0.4
0.6
0.8
1.0FPR
Federated Learning, Attacker#=0= 1.2, Atttacker#=0
Federated Learning, Atttacker#=5= 1.2, Atttacker#=5
Federated Learning, Atttacker#=27= 1.2, Atttacker#=27
Federated Learning, Atttacker#=54= 1.2, Atttacker#=54
= 1.5, Atttacker#=0
Figure 7: Ever-present attack - main task accuracy, backdoor accuracy & FPR vs. iterations.
attack against conventional secure aggregation [9] is shownin Fig. 9.
Experiment setup: We simulate a network of N = 1000users with non-i.i.d. training data by dividing 50,000 trainingimages using a Dirichlet distribution with hyperparameter 0.9as in [5]. We assume a 3×3 (heights × degree) tree whichyields 27 subgroups.
Metrics: We use the following evaluation criteria: (1) De-tection Rate (DR) : number of rounds with attackers detected/number of rounds with attackers presented, (2) CorrectionRate (CR) : number of attackers in subgroups that have beenlabeled as malicious by the server/ total number of attackers,(3) False positive rate (FPR) : number of benign subgroupsthat have been labeled as malicious by the server/ total numberof subgroups and (4) After Attack Accuracy: main /backdooraccuracy after the attack
(a) (b) (c)
Figure 8: Examples of Backdoor data. Cars with certain attributesare classified as birds. (a) Cars painted in green; (b) Cars withracing stripe and (c) Cars in front of vertical stripes background.
0 2000 4000 6000 8000 10000 12000 140000.00
0.25
0.50
0.75
1.00One-shot attack
0 2000 4000 6000 8000 10000 12000 140000.00
0.25
0.50
0.75
1.00Continuous attack
Green, Main taskGreen, Backdoor
Background, Main taskBackground,Backdoor
Stripe, Main taskStripe, Backdoor
Local Parameter Scale Factor
Figure 9: After-attack accuracy of three different backdoor datasets of [5] versus the scale factor. 1000 users in total and 2.7%malicious.
Performance under ever-present attack: In Fig. 7 weconsider the attack that lasts from the very beginning of train-ing to the convergence time, a.k.a. "ever-present attack". Weshow the main /backdoor accuracy and FPR as a function ofthe number of iterations of training by considering differentattacker numbers and thresholds. When no attacker exists, ourprotocol achieves almost the same training efficiency as con-ventional federated learning. By comparing our protocol withdifferent threshold expansion factors ρ, we note that a tighter
threshold will cause false positives occasionally, but the over-all training efficiency is not influenced. When ever-presentattackers are presented, although the attackers can easily in-ject backdoor to the global model, the training process is stag-nated because the attacker will eliminate updates from benignusers as shows in (1). On the other hand, when the numberof attackers is smaller than the number of subgroups (e.g.,attacker# = 5), our protocol can eliminate all attacker exceptsfor first few rounds when the standard deviation of updatesfrom benign users is high, and the early injected backdoorwill be "forgotten" while the main task converges normally.Although our protocol cannot avoid backdoor injection whena large number of ever-present attackers are presented, the at-tackers cannot achieve the attack goal either due to the failureof main task convergence.
Performance against converging phase attacker: [35][5] concludes that launching the attack toward the conver-gence time is beneficial to attackers because the backdoortends to be "forgotten" easily in early stages and the main taskmay be jeopardized, which consists with our results above.In Fig. 10, we show the performance of our protocol againstthe one-shot attacker (attacking at the last iteration of con-vergence) and continuous attacker (continuously attackingfor 5 iterations before convergence) launched in the converg-ing phase. Our protocol achieves 100% detection rate, near100% correction rate, and 0% false positive rate in most ofthe scenarios, which supports our analysis in Section 5.2 thatthere will be significant distribution disparity between benignsubgroup and subgroups with attackers. After-attack accu-racy shows that neither the main task was influenced nor thebackdoor was injected. However, when the number of attack-ers exceeds the number of subgroups, the training process isstagnated because most of the subgroups will be replaced.
8 Related Work
8.1 Model-Poisoning Attacks to DistributedMachine Learning
In distributed machine learning users and the aggregationserver, if any, do not have access to the training data sets pos-sessed by other users. This provides the opportunity for vari-ous malicious attacks. Data poisoning [7] and model replace-ment [5] are two most common attacks that aim at generating
10
0 2000 4000 6000 8000 10000 12000 140000.00
0.25
0.50
0.75
1.00Green car
0 2000 4000 6000 8000 10000 12000 140000.00
0.25
0.50
0.75
1.00Car with tripe
0 2000 4000 6000 8000 10000 12000 140000.00
0.25
0.50
0.75
1.00Car in stripe background
DR, One TimeDR, ContinuousCR, One TimeCR, ContinuousFPR, One TimeFPR, Continuous
(a) DR, CR and FPR vs. local parameter scale factor.
0 20 40 60 80 100 120 1400.00
0.25
0.50
0.75
1.00Green car
0 20 40 60 80 100 120 1400.00
0.25
0.50
0.75
1.00Car with tripe
0 20 40 60 80 100 120 1400.00
0.25
0.50
0.75
1.00Car in stripe background
DR, One TimeDR, ContinuousCR, One TimeCR, ContinuousFPR, One TimeFPR, Continuous
(b) DR, CR and FPR vs. Number of attackers.
0 2000 4000 6000 8000 10000 12000 140000.00
0.25
0.50
0.75
Green car
0 2000 4000 6000 8000 10000 12000 140000.00
0.25
0.50
0.75
Car with tripe
0 2000 4000 6000 8000 10000 12000 140000.00
0.25
0.50
0.75
Car in stripe background
Main task, One TimeBackdoor, One TimeMain task, ContinuousBackdoor, Continuous
(c) After attack accuracy vs. local parameter scale factor.
0 20 40 60 80 100 120 1400.00
0.25
0.50
0.75
Green car
0 20 40 60 80 100 120 1400.00
0.25
0.50
0.75
Car with tripe
0 20 40 60 80 100 120 1400.00
0.25
0.50
0.75
Car in stripe background
Main task, One TimeBackdoor, One TimeMain task, ContinuousBackdoor, Continuous
(d) After attack accuracy vs. Number of attackers.
Figure 10: Performance of our protocol (ρ = 1.2) against backdoor attack with different backdoor trigger.
malicious global models at the attacker’s will. Specifically,traditional data poisoning attacks [12, 18, 25], which usuallytarget at cloud-centric learning, can influence the behaviorof the global model by constructing poisoning samples anduploading poisoned features. According to recent study, theattacker needs to pollute about 20% to 40% of the trainingdata in targeted classes [7] or focus on training data withrare features [21] in order to launch data poisoning attacks.In large-scale machine learning the attacker usually needsto compromise 10% to 50% of the participants [8, 31] whocontinuously upload malicious models.
Model replacement attackers, on the other hand, leveragethe information of the global model and locally construct ma-licious inputs that modify the global model precisely in theway they desire. To make the attack more effective, attackerscan adjust and augment their local model parameters to dom-inate the global model during the aggregation process. Ascompared to data poisoning attacks, model poisoning is moreefficient and effective. The objective of the attack, even if itis to completely replace the global model, can be achieved inone shot with one or few attackers. As sufficient validationdatasets are not always available to the aggregator, detec-tion of such attacks is nontrivial. Existing model replacementdetection techniques like Byzantine-robust federated learn-ing [8] address the problem by evaluating the consistency ofmodels provided by all participating users, either by ampli-tudes [8] or by angles [15] of the received gradient updates.As these techniques need to evaluate models provided byindividual users at each iteration, a prohibiting complexity
is introduced in large-scale systems with many participants.Moreover, all the above detection techniques require access toplaintext of individual models which is not available in secureaggregation.
8.2 Privacy Preserving Machine LearningPrivacy-preserving machine learning [4] aims to prevent orconstrain disclosure of training data or models to unauthorizedparties. To this end various techniques have been proposedwhich can be roughly categorized into secure multi-partycomputation (MPC) and differential privacy, based on theunderlying techniques they employ. In the area of distributedmachine learning, existing MPC-based proposals usually relyon heavy cryptographic primitives include gable circuits, par-tially or fully homomorphic encryption, oblivious transfer, etc.Recently, promising progresses have been made toward smallnetworks especially for inference tasks [10, 22, 24, 27–30, 33].However, there has yet been a practical cryptographic toolthat supports efficient training of complex models, e.g., fordeep learning tasks.
Differential privacy (DP), on the other hand, focuses onpublishing aggregated information with limited disclosureof private information. For example, one approach [3, 16]is to protect data/model/outputs by allowing users to addzero-mean statistical noise (e.g., Laplace noise) to make data/model/outputs indistinguishable and aggregate data/model/outputs with affordable variance. To maintain a desired accu-racy, one needs to carefully design the random noise without
11
degrading the level of privacy protection when the DP mecha-nism is repeated in the training process. As a result, it remainsa challenge in DP to maintain an appropriate trade-off be-tween privacy and model quality ( in terms of accuracy losscaused by added noise) especially in deep learning tasks.
9 Conclusion
This paper presented a novel secure aggregation scheme forfederated learning, which supports backdoor detection andsecure aggregation simultaneously with oblivious randomgrouping and partial parameter disclosure. Compare to con-ventional secure aggregation protocol, our protocol reducescomputation complexity from O(N2 +mN) (at user side) andO(mN2) (at server-side) to O(N +n2 +m) and O(mN +nN),respectively, where N is the total number of users, n the num-ber of users in subgroups and m the size of the model. Wevalidated our design through experiments with 1000 simulatedusers. Experimental results demonstrate the efficiency andscalability of the proposed design. Moreover, our protocol uti-lizes adaptive outlier detection to address potential backdoorattack in federated learning. Experimental evaluation on state-of-art adaptive backdoor attack attack [5] combining withpossible attack strategies such as continuous attacks showthat our protocol can significantly improve the robustnessagainst backdoor attacks.
10 Acknowledgement
This project is partially supported by NSF grantECCS#1923739.
References
[1] . Federated learning: Collaborative machinelearning without centralized training data.https://ai.googleblog.com/2017/04/federated-learning-collaborative.html. [Online; accessed June-2019].
[2] . Under the hood of the pixel 2: How ai is supercharg-ing hardware. https://ai.google/stories/ai-in-hardware/.[Online; accessed June-2019].
[3] ABADI, M., CHU, A., GOODFELLOW, I., MCMAHAN,H. B., MIRONOV, I., TALWAR, K., AND ZHANG, L.Deep learning with differential privacy. In Proceedingsof the 2016 ACM SIGSAC Conference on Computer andCommunications Security (2016), ACM, pp. 308–318.
[4] AL-RUBAIE, M., AND CHANG, J. M. Privacy pre-serving machine learning: Threats and solutions. IEEESecurity and Privacy Magazine 17, 2 (2019), 49–58.
[5] BAGDASARYAN, E., VEIT, A., HUA, Y., ESTRIN, D.,AND SHMATIKOV, V. How to backdoor federated learn-ing. arXiv preprint arXiv:1807.00459 (2018).
[6] BHAGOJI, A. N., CHAKRABORTY, S., MITTAL, P.,AND CALO, S. Analyzing federated learning throughan adversarial lens. In International Conference onMachine Learning (2019), pp. 634–643.
[7] BIGGIO, B., NELSON, B., AND LASKOV, P. Poisoningattacks against support vector machines. arXiv preprintarXiv:1206.6389 (2012).
[8] BLANCHARD, P., GUERRAOUI, R., STAINER, J.,ET AL. Machine learning with adversaries: Byzantinetolerant gradient descent. 119–129.
[9] BONAWITZ, K., IVANOV, V., KREUTER, B., MARCE-DONE, A., MCMAHAN, H. B., PATEL, S., RAMAGE,D., SEGAL, A., AND SETH, K. Practical secure ag-gregation for privacy-preserving machine learning. InProceedings of the 2017 ACM SIGSAC Conference onComputer and Communications Security (2017), ACM,pp. 1175–1191.
[10] BOST, R., POPA, R. A., TU, S., AND GOLDWASSER,S. Machine learning classification over encrypted data.In NDSS (2015), vol. 4324, p. 4325.
[11] CHAN, T.-H. H., SHI, E., AND SONG, D. Privacy-preserving stream aggregation with fault tolerance. InInternational Conference on Financial Cryptographyand Data Security (2012), Springer, pp. 200–214.
[12] CHEN, X., LIU, C., LI, B., LU, K., AND SONG, D.Targeted backdoor attacks on deep learning systemsusing data poisoning. arXiv preprint arXiv:1712.05526(2017).
[13] DIFFIE, W., AND HELLMAN, M. New directions incryptography. IEEE transactions on Information Theory22, 6 (1976), 644–654.
[14] DOUCEUR, J. R. The sybil attack. In Internationalworkshop on peer-to-peer systems (2002), Springer,pp. 251–260.
[15] FUNG, C., YOON, C. J., AND BESCHASTNIKH, I. Mit-igating sybils in federated learning poisoning. arXivpreprint arXiv:1808.04866 (2018).
[16] GEYER, R. C., KLEIN, T., AND NABI, M. Differen-tially private federated learning: A client level perspec-tive. arXiv preprint arXiv:1712.07557 (2017).
[17] GORYCZKA, S., AND XIONG, L. A comprehensivecomparison of multiparty secure additions with differen-tial privacy. In IEEE Transactions on Dependable andSecure Computing (2015), IEEE.
12
[18] GU, T., DOLAN-GAVITT, B., AND GARG, S. Bad-nets: Identifying vulnerabilities in the machine learningmodel supply chain. arXiv preprint arXiv:1708.06733(2017).
[19] GUPTA, S., AGRAWAL, A., GOPALAKRISHNAN, K.,AND NARAYANAN, P. Deep learning with limited nu-merical precision. In International Conference on Ma-chine Learning (2015), pp. 1737–1746.
[20] HE, K., ZHANG, X., REN, S., AND SUN, J. Deepresidual learning for image recognition. In Proceedingsof the IEEE conference on computer vision and patternrecognition (2016), pp. 770–778.
[21] HUANG, L., JOSEPH, A. D., NELSON, B., RUBIN-STEIN, B. I., AND TYGAR, J. D. Adversarial machinelearning. In Proceedings of the 4th ACM workshop onSecurity and artificial intelligence (2011), pp. 43–58.
[22] JUVEKAR, C., VAIKUNTANATHAN, V., AND CHAN-DRAKASAN, A. Gazelle: A low latency framework forsecure neural network inference. In 27th USENIX Secu-rity Symposium (USENIX Security 18) (2018), USENIX.
[23] KONECNY, J., MCMAHAN, H. B., YU, F. X.,RICHTÁRIK, P., SURESH, A. T., AND BACON, D. Fed-erated learning: Strategies for improving communica-tion efficiency. arXiv preprint arXiv:1610.05492 (2016).
[24] LIU, J., JUUTI, M., LU, Y., AND ASOKAN, N. Oblivi-ous neural network predictions via minionn transforma-tions. In 2017 ACM SIGSAC Conference on Computerand Communications Security (CCS ’17) (2017), ACM.
[25] LIU, Y., MA, S., AAFER, Y., LEE, W.-C., ZHAI, J.,WANG, W., AND ZHANG, X. Trojaning attack on neuralnetworks.
[26] MCMAHAN, B., MOORE, E., RAMAGE, D., HAMP-SON, S., AND Y ARCAS, B. A. Communication-efficient learning of deep networks from decentralizeddata. In Artificial Intelligence and Statistics (2017),PMLR, pp. 1273–1282.
[27] MOHASSEL, P., AND RINDAL, P. Aby3: A mixed pro-tocol framework for machine learning. In 2018 ACMSIGSAC Conference on Computer and CommunicationsSecurity (CCS ’18) (2018), ACM.
[28] MOHASSEL, P., AND ZHANG, Y. Secureml: A systemfor scalable privacy-preserving machine learning. In2017 IEEE Symposium on Security and Privacy (SP)(2017), IEEE, pp. 19–38.
[29] NIKOLAENKO, V., WEINSBERG, U., IOANNIDIS, S.,JOYE, M., BONEH, D., AND TAFT, N. Privacy-preserving ridge regression on hundreds of millions of
records. In 2013 IEEE Symposium on Security andPrivacy (2013), IEEE, pp. 334–348.
[30] RIAZI, M. S., WEINERT, C., TKACHENKO, O.,SONGHORI, E. M., SCHNEIDER, T., AND KOUSHAN-FAR, F. Chameleon: A hybrid secure computation frame-work for machine learning applications. In 2018 AsiaConference on Computer and Communications Security(ASIACCS’18) (2018), ACM.
[31] SHEN, S., TOPLE, S., AND SAXENA, P. A uror: de-fending against poisoning attacks in collaborative deeplearning systems.
[32] SHI, E., CHAN, H., RIEFFEL, E., CHOW, R., ANDSONG, D. Privacy-preserving aggregation of time-seriesdata. In Annual Network & Distributed System SecuritySymposium (NDSS) (2011), Citeseer.
[33] WAGH, S., GUPTA, D., AND CHANDRAN, N. Se-cureNN: Efficient and private neural network training.In PETS 2019 (2019).
[34] WALPOLE, R. E., MYERS, R. H., MYERS, S. L., ANDYE, K. Probability and statistics for engineers andscientists, vol. 5. Macmillan New York, 1993.
[35] XIE, C., HUANG, K., CHEN, P.-Y., AND LI, B. Dba:Distributed backdoor attacks against federated learning.In International Conference on Learning Representa-tions (2019).
[36] ÁCS, G., AND CASTELLUCCIA, C. I have a dream!(differentially private smart metering). In 2011 In-ternational Workshop on Information Hiding (2011),Springer.
A Proof of Theorem 2
Theorem 2: Let {X ′i }i∈[1,n] be n samples randomly se-lected from {Xi}i∈[1,N] in Theorem 1 and Y = y ∈ [E(X ′)−RH/2,E(X ′) + RH/2], where E(X ′) is the mean vector of
{X ′i }i∈[1,n]. If RH = 2(1−√
n−1n )ε. The expectation of the
bound of P(|Xi− y|< σ|Y ) is σ2N
ε2 .
Proof. Assume {X ′i }i∈1→n have variance σ2n. Using Cheby-
shev Inequality, we can gets:
P(||Xi−E(X ′)|| ≥ ε)≤ σ2n
ε,ε2 >
RH
2
13
Combining with Y = y ∈ [E(X ′)−RH/2,E(X ′)+RH/2], wecan derive the bounds:
P(||Xi− y|| ≥ ε|y ∈ [E(X ′)−RH/2,E(X ′)+RH/2])
≤ σ2n
(ε− RH2 )2
,ε >RH
2
Because {X ′i }i∈1→n are randomly sampled from {Xi}i∈1→N ,according to the distribution of sample variance [34], theexpectation of σ
2n is n−1
n σ2N . Therefore the expectation of the
bounds can be written as:
E(σ
2n
(ε− RH2 )2
) =(n−1)σ2
N
n(ε− RH2 )2
When we let RH = 2(1−√
n−1n )ε the expectation will be σ
2N
ε2 ,which is the same in Theorem 1.
B Type I Attacker
Here we give the formal proof of Theorem 3. We have the fol-lowing parameters setting in our protocol execution: a securityparameter k which the underlying cryptographic primitivesare instantiated with, a threshold t to limit the number of cor-rupted parties, a server S and a user set U which contains Nusers in total, each one is denoted as ui where i = {1,2, ...,N}.During the execution, we require that the total dropouts tobe limited by certain threshold. We denote the input of eachuser u as xu and xU ′ = {xu}u∈U ′ as the inputs of any subset ofusers U ′ ⊆U .
In the protocol execution process, the view of a party con-sists of its inputs and the randomness and all messages thisparty received from other parties. If a party drop out during theprocess, its view remains the same as that before the dropoutoccurs.
Given any subset C ⊆ U ∪ {S} of the parties, letRealU,t,k
C (xU,U) be a random variable which represents thejoint view of all parties in C which includes some honest butcurious users and an honest but curious server who can com-bine knowledge of these users. Our theorem shows that thejoint view of any subset of honest but curious parties can besimulated given the inputs of these part of users and only thesum of the input from those users within the same group. Inparticular, we prove that with a certain threshold t, the jointview of the server and any set of less than t corrupted partywill not leak any information of other honest parties besidesthe output which is known to all the parties and server.
Theorem 3 (Local Model Privacy under Type I Attackers)There exists a PPT simulator SIM such that for all t,U,xUand C ⊆ U ∪ {S}, where |C\{S}| ≤ t, the output of SIMis computationally indistinguishable from the output of
RealU,t,kC (xU,U)
RealU,t,kC (xU,U)≈ SIMU,t,k
C (xC,z,U)
where
z =
{∑u∈U\C xu if |U| ≥ t⊥ o.w.
Proof. We prove the theorem by a standard hybrid argument.We will define a simulator SIM through a series of subsequentmodifications to the random variable Real, so that any twoconsecutive random variables are computationally indistin-guishable.
Hybrid0 This random variable is distributed exactly asReal, the joint view of the parties C in a real execution ofthe protocol.
Hybrid1 In this hybrid, we change the behavior of thosehonest parties in simulator. Instead of using the secret sharingkeys (CSK
u ,CPKv ) produced by Diffie-Hellman key exchange,
these parties use a uniformly random encryption key cu,vchosen by the simulator. This hybrid is computationally indis-tinguishable with the previous one guaranteed by our ServerInvolved Decisional Diffie-Hellman key exchange.
Recall that in our Server Involved D-H key exchange, theonly modification we made compared to the D-H key ex-change in original protocol is that, we add a randomness to thepublic key. In specific, instead of exchanging the public key{sPK
u ,sPKv }, user u and v will exchange {(sPK
u )ru,v ,(sPKv )ru,v}.
We show that for any honest but curious parties who is giventhis pair of public key can not distinguish the secret su,v com-puted by these keys from a uniformly random string.
Assume there exists a PPT algorithm A which can breakthe server involved D-H key exchange with advantage ε, thenthere exists an a PPT algorithm B which can break DDHassumption with the same advantage. The proof follows thesetting in original protocol. Consider a multiplicative cyclicgroup G of order q, and with generator g, for a,b,c uniformlyrandom chosen in Zq and a randomness ra,b produced by V RF ,A can distinguish gabra,b from gc, then for B , it times the resultof D-H key exchange for ra,b times, that is (gab)ra,b and theninvoke A , it breaks the DDH with the same advantage.
Hybrid2 In this hybrid, we change the ciphertexts en-crypted by the honest parties. Instead of using the encryptionof correct shares SSK
u and bu, each honest party just encrypt0∗ which has the same length as those shares and send it toother honest parties. This hybrid is computationally indistin-guishable with the previous one. This is guaranteed by theIND-CPA security of secret-sharing scheme.
Hybrid3 In this hybrid, we change the shares of bu givento the corrupted parties to shares of 0. Note that in our design,the view of the corrupted parties contains no more than dt/neshares of each bu,since the honest users do not reveal bu forcorrupted parties during Unmasking procedure. This hybridis distributed the same as the previous one. This is guaranteedby the properties of Shamir’s secret sharing scheme.
14
Hybrid4 In this hybrid, we change a part of pariwise mask-ing information of all the parties in U . Instead of computingPRG(bu), the simulator use a uniformly random vector of thesame size. Note that in previous hybrid, since bu is uniformlyrandom and is substituted by shares of 0,the output does notdepend on the seed of PRG. Thus, when we substitute theresult of PRG with a random value. the result is guaranteedto have the same distribution by the security of PRG whichmeans this hybrid is distributed the same as the previous one.
Hybrid5 In this hybrid,we substitute the masked input with-out the input of the users. That is,instead of sending
yu = xu +PRG(bu)+ ∑∀v∈Gs
u:u<vPRG(su,v)−
∑∀v∈Gs
u:u>vPRG(sv,u)+ ∑
∀v∈Gpu :u<v
(ΛRH ∧PRG(su,v))−
∑∀v∈Gp
u :u>v
(ΛRH ∧PRG(sv,u)) (mod R) (6)
users send:
yu = PRG(bu)+ ∑∀v∈Gs
u:u<vPRG(su,v)−
∑∀v∈Gs
u:u>vPRG(sv,u)+ ∑
∀v∈Gpu :u<v
(ΛRH ∧PRG(su,v))−
∑∀v∈Gp
u :u>v
(ΛRH ∧PRG(sv,u)) (mod R) (7)
Recall that in the previous hybrid, PRG(bu) was changedto be uniformly random, so xu +PRG(bu) is also uniformlyrandom. Thus, this hybrid is identical to the previous oneand further more, all the following hybrids do not depend thevalues of xu
Hybrid6 In this hybrid, we change the behavior by sending0 instead of SSK
u generated by the honest parties to all otherparties. Similar with Hybrid3,the properties of Shamir’s secretsharing scheme guarantee that this hybrid is identical to theprevious one.
Hybrid7In this hybrid, for a specific honest user u′, in orderto compute yu, we substitute the key su,v and sv,u for all otherusers in Gp
u with two uniformly random value. This hybrid iscomputationally indistinguishable with the previous one. Thisis guaranteed by the Decisional Diffie-Hellman assumption.
Hybrid8 In this hybrid, instead of using PRG(su,v) andPRG(sv,u) in ΛRH ∧PRG(su,v) and ΛRH ∧PRG(sv,u), we com-pute yu with two uniformly random values with the same size.Recall that The bits of ΛRH ’s elements are 0 if they are inrange [RH ,RU ], other wise the bits are 1. This operation willnot affect the randomness since all the values are doing an andoperation with 0 on high bits, this hybrid is indistinguishablewith the previous one. This is guaranteed by the security ofPRG.
Hybrid9 In this hybrid, for a specific honest user u′, in orderto compute yu, we substitute the key su,v and sv,u for all otherusers in Gs
u with two uniformly random value. This hybrid is
computationally indistinguishable with the previous one. Thisis guaranteed by the Decisional Diffie-Hellman assumption.
Hybrid10 In this hybrid, instead of using PRG(su,v) andPRG(sv,u), we compute yu with two uniformly random valueswith the same size. Similar to Hybrid4, this hybrid is distin-guishable with the previous one. This is guaranteed by thesecurity of PRG.
Hybrid11 In this hybrid, for all the honest users,instead ofsending
yu = xu +PRG(bu)+ ∑∀v∈Gs
u:u<vPRG(su,v)−
+ ∑∀v∈Gs
u:u>vPRG(sv,u)+ ∑
∀v∈Gpu :u<v
(ΛRH ∧PRG(su,v))−
∑∀v∈Gp
u :u>v
(ΛRH ∧PRG(sv,u)) (mod R) (8)
users send
yu = wu +PRG(bu)+ ∑∀v∈Gs
u:u<vPRG(su,v)−
∑∀v∈Gs
u:u>vPRG(sv,u)+ ∑
∀v∈Gpu :u<v
(ΛRH ∧PRG(su,v))−
∑∀v∈Gp
u :u>v
(ΛRH ∧PRG(sv,u)) (mod R) (9)
where wu is uniformly random and satisfy ∑u∈U\C
wu =
∑u∈U\C
xu = z. This hybrid is identically distributed as previous
one.Thus, we can define a PPT simulator SIM as the last hybrid
describe and the output of this simulator is computationallyindistinguishable from the output of Real.
In the next part we prove that for our random tree structureunder honest but curious setting, the final order output by theprotocol is indistinguishable from a uniformly random valuein the co-domain of selected hash function.
Theorem 4 (Random Tree Structure Secrecy) There existsa PPT simulator SIM such that for all t,U,xU and C⊆U∪{S},where |C\{S}| ≤ t, the output of SIM is indistinguishablefrom the output of real protocol:
RealU,t,kC (Ru,RS,CPK
u )≈ SIMU,t,kC (Ru,RS,CPK
u )
Proof. we show this by a standard hybrid argument. We willdefine a simulator SIM through a series of modifications tothe random variable Real, so that any two consecutive randomvariables are indistinguishable.
Hybrid0 This hybrid is exactly the same as Real.Hybrid1 In this hybrid, we fix an specific honest user u′
and change its behavior. Instead of sending cPKu , u′ sends a
uniformly random key with the appropriate size to the server.Notice that Idu = HASH(Rs||cPK
u ||Ru), thus this hybrid is dis-tributed exactly the same as the previous one. This is guaran-teed by the properties of hash function.
15
Hybrid2 In this hybrid, for the specific user defined in pre-vious hybrid, the final output HASH( ∑
∀v∈G:v 6=uIdv) is indistin-
guishable from the previous one. It is easy to show that ifthere exists one more honest user except u′, the result willguarantee to uniformly distribute by the properties of hashfunction.
Moreover, we can prove that the result still holds for ma-licious users and compromised server under the security ofthe commitment protocol. After commitment, these corruptedusers and server could not make any change on Rs and Ru,and then the result of Idu is indistinguishable from a randomvalue in co-domain of hash function by the properties of hashfunction.
C Type II Attacker
In this part we give a formal statement to our claim. For afixed attacker A in a certain subgroup, we show that even ifthere is another attacker B existing in the same subgroup, theycan not know they are in the same subgroup which means thatthe joint view of A and B is indistinguishable from the viewof A .
Theorem 5 (Indistinguishability of Type II attackers inthe same subgroup) For any type II attacker A in a certainsubgroup, if there is another type II attacker B in the samesubgroup but is not peered to A , ViewA ,B , the joint view ofA and B is indistinguishable from ViewA , the view of A :
ViewA ≈ViewA ,B
Proof. We prove the theorem by a standard hybrid argument.Recall that there is n users in a subgroup and each user canonly pairwise mask with previous and next κ users accordingto Idu in our circle-like topology design. We first fix theposition of a given isolated attacker A which is peered with κ
honest user in this subgroup and an unassigned attacker B .Hybrid0 This hybrid is exactly the same as our setting. In
this hybrid, the view of attacker A contains the transcriptproduced by running the protocol with all the pairwise maskpeers of A and all the key pairs owned by the attackers besidesthe key pairs shared by the secret sharing scheme since weallows the cooperation of attackers.
Hybrid1 In this hybrid, we substitute all the information ofthese κ honest user with random strings and the view of Aremains the same. This part of proof follows the same as ourproof for type I attacker and we omit it.
Hybrid2 In this hybrid, for a specific user u who is not apeer of A and let v denote the pairwise mask peers of u, wesubstitute the key cSK
u with cSKB . This modification will not
change the view of A since A has no information about cSKu
guaranteed by the fact that the server will only send the thepublic key from user u’s pairwise companions.
Hybrid3 In this hybrid, instead of using sSKu and bu, user u
shares the share of B . Similar with Hybrid2, the server willonly relay the encrypted secret shares to the peers of user u.Thus, for attacker A , the view remains the same.
Hybrid4 In this hybrid, we substitute the pairwise key pairs(sSK
v ,sPKu ) by the keys of B . Similarly, this will not change the
view of A . The view of A has already contains the key of Band the circle-like topology guarantee that A cannot executethe pairwise masking with u so they share no informationabout the pairwise mask keys. Till now, we have substitute allthe information of an honest user u by the information of anattacker B . The joint view of A and B contains the transcriptsproduced by running the protocols. Since we have alreadyprove that the protocol is privacy-preserving. The joint viewof A and B makes no difference with joint view of A and anhonest user u. Thus, if two attackers are not peered with eachother for pairwise masking, any of them cannot get any extrainformation more than the information it owns.
D Performance analysis: THEORETICAL
In the following analysis, we use h and d to indicate the heightand degree of the tree structure, respectively.
User’s Computational Cost: O(N + n2 +m). Each useru’s computational cost includes: (1) performing n+ 2(κ+log N
n ) key agreements of complexity O(n); (2) creationof t-out-of-n Shamir’s secret shares of sSK
u and bu, whichis O(n2); (3) verification of random full order generation,which is O(N); and (4) performing intra-group masking andinter-group masking on data vector xu, which is O(m(κ +h)) = O(m). The overall computational cost for each user isO(N +n2 +m).
User’s Communication Cost: O(N +m). Each user u’scommunication cost includes: (1) exchanging keys with allother users in the verification phase, which accounts for O(N);(2) transmitting constant parameters like Ru which is O(1);(3) sending and receiving 2(n−1) encrypted secret shares ofconstant size, which is O(n); (4) sending a masked data vectorof size m to the server, which is O(m); (5) sending the servern secrete shares, which is O(n). The overall communicationcost for each user is O(N +m).
Server’s Computational Cost: O(mN + nN). Theserver’s computational cost includes: (1) reconstructing Nt-out-of-n Shamir’s secrets, which takes a total time of O(Nn)(Note that for each user in the same subgroup, the Lagrangebasis polynomials for reconstructing Shamir’s secrets remainthe same.); (2) random full order generation, which is O(N);(3) for each dropped user, the server needs to unmasking itsmask for all survived users in the same group. The expectedtime cost yields to O(md(κ+h)), where d is the number ofdropped users. For each survived user, the server needs tounmasking its own mask. And the time cost is O(m(N−d)).The overall computational cost for the server is O(mN +nN).Server’s communication Cost: O(N2 +mN). The server’s
16
100 150 200 250 300 350 400 450 500#Clients
50000
100000
150000
200000
Serv
er R
unin
g Ti
me
(ms)
Original, 0% dropoutOriginal, 10% dropoutOriginal, 20% dropoutOriginal, 30% dropoutOurs, 0% dropoutOurs, 10% dropoutOurs, 20% dropoutOurs, 30% dropout
(a) Running time for the server in different clients number. The datavector size is fixed to 100K entries.
100000 150000 200000 250000 300000 350000 400000 450000 500000Data Vector Size
100000
200000
300000
400000
500000
600000
700000
Serv
er R
unin
g Ti
me
(ms)
Original, 0% dropoutOriginal, 10% dropoutOriginal, 20% dropoutOriginal, 30% dropoutOurs, 0% dropoutOurs, 10% dropoutOurs, 20% dropoutOurs, 30% dropout
(b) Running time for the server, as the size of the data vector increases.The number of clients is fixed to 500.
Figure 11: Server Running Time. Solid line represent our Tree BasedSecure Aggregation Protocol. Dash line represent original SecureAggregation Protocol
communication cost is mainly dominated by (1) its media-tion of pairwise communications between the users, which isO(N2) and (2) receiving masked data vectors from each user,which is O(mN). The overall communication is O(N2 +mN).
E Performance analysis: Simulation
To evaluate the performance of our protocol, we test our proto-col in different tree structures with 1000 and 1500 participantsunder a 15% drop rate. Comparison of our protocol with theoriginal secure aggregation protocol is presented in AppendixE. The experiments are performed on a Windows desktop withIntel i5-8400 (2.80 Ghz), with 16 GB of RAM. We assumethat users will only drop out of the protocol after sending theirsecret shares to other users in the same subgroup, but beforesending masked input to the server. This is the "worst-case"dropout because the server needs to perform mask recoveryfor other users in that subgroup. As shown in Table 3, whenthe heights or degree of the tree structure increase, the user’srunning time and communication costs decrease significantly.Therefore, by adjusting the tree structure, SAFELearning isable to control its complexity by limiting the size of sub-groups, which makes it highly scalable.
We compare the overhead of our protocol with Bonawitzet. al. [9] (abbr. as “original") in the server and user side withsimulation experiments. In the experiment, we use 2×2 treestructure and each user u pair mask with previous and nextκ = 4 users.
Fig. 11 shows the server’s running time comparison be-tween two protocols when the number of clients and data vec-tor size increases. Note that, with 0 dropout rate, the server’srunning time is almost the same between two protocols. Thisis because as long as all pairwise masks are canceled out, theserver of both protocols only needs to unmask bu for each sur-vived user. But when there are the dropped users, the serverin the original protocol needs to unmask O(N) pairwise maskfor each dropped user where our protocol only needs to un-mask O(1). So, in Fig. 11, the running time growth rate of theoriginal protocol is increased significantly as the dropout raterises up when our protocol still keeps the same growth rate as0 dropout rate.
As seen in Fig. 12 (a), The users’ running time of both pro-tocol increases linearly when the number of users increases.But our protocol has a significantly lower increase rate be-cause the computation complexity of each user in our protocolis O(N + n2 +m) instead of O(N2 +mN). And because thenumber of pairwise masking needed in our protocol is con-stant for each user, the user’s computation cost won’t havesignificant change when the data vector size increase. So inFig. 12 (b), the running time’s increase speed of our protocolis imperceptible compare to the original protocol that needsN pairwise masking operations for each user. In Fig. 12 (c),the plot shows that both protocols’ communication cost aredominant by the data vector size, but our protocol still haslower communication expansion as the number of clients in-crease, although this increase is relatively small compared tothe impact of increasing the size of the data vector.
17
Table 3: users’ running time, total data transfer and server running time in different tree structure. The data vectorsize is fixed to 100K entries, each entry is 10 bytes and the drop rate is fixed to 15%.
#users Tree structure(heights×degree) Running time per user Total data transfer per user Server running time1000 2×2 1166 ms 1.51 MB 209283 ms1000 3×2 568 ms 1.40 MB 207076 ms1000 3×3 197 ms 1.33 MB 190610 ms1500 2×2 1877 ms 1.77 MB 312432 ms1500 3×2 899 ms 1.61 MB 310422 ms1500 3×3 298 ms 1.50 MB 287882 ms
100 150 200 250 300 350 400 450 500#Clients
0
500
1000
1500
2000
2500
3000
3500
Runi
ng T
ime
per C
lient
(ms)
Original, 0% dropoutOriginal, 10% dropoutOriginal, 20% dropoutOriginal, 30% dropoutOurs, 0% dropoutOurs, 10% dropoutOurs, 20% dropoutOurs, 30% dropout
(a) Running time per users, as the numberof clients increases. The data vector size isfixed to 100K entries
100000 200000 300000 400000 500000 600000Data Vector Size
2000
4000
6000
8000
Runi
ng T
ime
per C
lient
(ms)
Original, 0% dropoutOriginal, 10% dropoutOriginal, 20% dropoutOriginal, 30% dropoutOurs, 0% dropoutOurs, 10% dropoutOurs, 20% dropoutOurs, 30% dropout
(b) Running time per users, as the data vec-tor size increases. The number of users isfixed to 500
100 150 200 250 300 350 400 450 500#Clients
100
200
300
400
500
Tota
l Dat
a pe
r Clie
nt (K
B)
Original, 100K data entriesOriginal, 300K data entriesOriginal, 500K data entriesOurs, 100K data entriesOurs, 300K data entriesOurs, 500K data entries
(c) Total data transfer per client, as thenumber of clients increases. Assumes nodropouts.
Figure 12: Client Running Time. Solid line represent our Tree Based Secure Aggregation Protocol. Dash line represent original SecureAggregation Protocol
18