deepiot: compressing deep neural network structures ?· deepiot: compressing deep neural network...

Download DeepIoT: Compressing Deep Neural Network Structures ?· DeepIoT: Compressing Deep Neural Network ...…

Post on 05-Sep-2018




0 download

Embed Size (px)


  • DeepIoT: Compressing Deep Neural Network Structures forSensing Systems with a Compressor-Critic Framework

    Shuochao YaoUniversity of Illinois Urbana


    Yiran ZhaoUniversity of Illinois Urbana


    Aston ZhangUniversity of Illinois Urbana


    Lu SuState University of New York at


    Tarek AbdelzaherUniversity of Illinois Urbana


    ABSTRACTRecent advances in deep learning motivate the use of deep neutralnetworks in sensing applications, but their excessive resource needson constrained embedded devices remain an important impediment.A recently explored solution space lies in compressing (approximat-ing or simplifying) deep neural networks in some manner beforeuse on the device. We propose a new compression solution, calledDeepIoT, that makes two key contributions in that space. First,unlike current solutions geared for compressing specic types ofneural networks, DeepIoT presents a unied approach that com-presses all commonly used deep learning structures for sensingapplications, including fully-connected, convolutional, and recur-rent neural networks, as well as their combinations. Second, unlikesolutions that either sparsify weight matrices or assume linear struc-ture within weight matrices, DeepIoT compresses neural networkstructures into smaller dense matrices by nding the minimumnumber of non-redundant hidden elements, such as lters and di-mensions required by each layer, while keeping the performanceof sensing applications the same. Importantly, it does so using anapproach that obtains a global view of parameter redundancies,which is shown to produce superior compression. e compressedmodel generated by DeepIoT can directly use existing deep learn-ing libraries that run on embedded and mobile systems withoutfurther modications. We conduct experiments with ve dierentsensing-related tasks on Intel Edison devices. DeepIoT outperformsall compared baseline algorithms with respect to execution timeand energy consumption by a signicant margin. It reduces thesize of deep neural networks by 90% to 98.9%. It is thus able toshorten execution time by 71.4% to 94.5%, and decrease energyconsumption by 72.2% to 95.7%. ese improvements are achievedwithout loss of accuracy. e results underscore the potential ofDeepIoT for advancing the exploitation of deep neural networks onresource-constrained embedded devices.

    Permission to make digital or hard copies of all or part of this work for personal orclassroom use is granted without fee provided that copies are not made or distributedfor prot or commercial advantage and that copies bear this notice and the full citationon the rst page. Copyrights for components of this work owned by others than ACMmust be honored. Abstracting with credit is permied. To copy otherwise, or republish,to post on servers or to redistribute to lists, requires prior specic permission and/or afee. Request permissions from 17, Del, Netherlands 2017 ACM. 978-1-4503-5459-2/17/11. . .$15.00DOI: 10.1145/3131672.3131675

    ACM Reference format:Shuochao Yao, Yiran Zhao, Aston Zhang, Lu Su, and Tarek Abdelzaher.2017. DeepIoT: Compressing Deep Neural Network Structures for SensingSystems with a Compressor-Critic Framework. In Proceedings of SenSys 17,Del, Netherlands, November 68, 2017, 14 pages.DOI: 10.1145/3131672.3131675

    1 INTRODUCTIONis paper is motivated by the prospect of enabling a smarter

    and more user-friendly category of every-day physical objects ca-pable of performing complex sensing and recognition tasks, such asthose needed for understanding human context and enabling morenatural interactions with users in emerging Internet of ings (IoT)applications.

    Present-day sensing applications cover a broad range of areasincluding human interactions [27, 59], context sensing [6, 34, 39,46, 54], crowd sensing [55, 58], object detection and tracking [7,29, 42, 53]. e recent commercial interest in IoT technologiespromises a proliferation of smart objects in human spaces at a muchbroader scale. Such objects will ideally have independent means ofinteracting with their surroundings to perform complex detectionand recognition tasks, such as recognizing users, interpreing voicecommands, and understanding human context. e paper exploresthe feasibility of implementing such functions using deep neuralnetworks on computationally-constrained devices, such as Intelssuggested IoT platform: the Edison board1.

    e use of deep neural networks in sensing applications has re-cently gained popularity. Specic neural network models have beendesigned to fuse multiple sensory modalities and extract temporalrelationships for sensing applications. ese models have shownsignicant improvements on audio sensing [31], tracking and local-ization [8, 38, 52, 56], human activity recognition [37, 56], and useridentication [31, 56].

    Training the neural network can occur on a computationallycapable node and, as such, is not of concern in this paper. e keyimpediment to deploying deep-learning-based sensing applicationslies in the high memory consumption, execution time, and energydemand associated with storing and using the trained network onthe target device. is leads to increased interest in compressingneural networks to enable exploitation of deep learning on low-endembedded devices.

    We propose DeepIoT that compresses commonly used deep neu-ral network structures for sensing applications through deciding









    ] 2

    2 N

    ov 2


  • SenSys 17, November 68, 2017, Del, Netherlands Shuochao Yao, Yiran Zhao, Aston Zhang, Lu Su, and Tarek Abdelzaher

    the minimum number of elements in each layer. Previous illumi-nating studies on neural network compression sparsify large denseparameter matrices into large sparse matrices [4, 22, 24]. In contrast,DeepIoT minimizes the number of elements in each layer, whichresults in converting parameters into a set of small dense matrices.A small dense matrix does not require additional storage for elementindices and is eciently optimized for processing [19]. DeepIoTgreatly reduces the eort of designing ecient neural structures forsensing applications by deciding the number of elements in eachlayer in a manner informed by the topology of the neural network.

    DeepIoT borrows the idea of dropping hidden elementsfrom a widely-used deep learning regularization method calleddropout [44]. e dropout operation gives each hidden elementa dropout probability. During the dropout process, hidden ele-ments can be pruned according to their dropout probabilities. ena thinned network structure can be generated. However, thesedropout probabilities are usually set to a pre-dened value, suchas 0.5. Such pre-dened values are not the optimal probabilities,thereby resulting in a less ecient exploration of the solution space.If we can obtain the optimal dropout probability for each hiddenelement, it becomes possible for us to generate the optimal slim net-work structure that preserves the accuracy of sensing applicationswhile maximally reducing the resource consumption of sensing sys-tems. An important purpose of DeepIoT is thus to nd the optimaldropout probability for each hidden element in the neural network.

    Notice that, dropout can be easily applied to all commonly usedneural network structures. In fully-connected neural networks,neurons are dropped in each layer [44]; in convolutional neuralnetworks, lters are dropped in each layer [14]; and in recurrentneural networks, dimensions are reduced in each layer [15]. ismeans that DeepIoT can be applied to all commonly-used neuralnetwork structures and their combinations.

    To obtain the optimal dropout probabilities for the neural net-work, DeepIoT exploits the network parameters themselves. Fromthe perspective of model compression, a hidden element that isconnected to redundant model parameters should have a higherprobability to be dropped. A contribution of DeepIoT lies in ex-ploiting a novel compressor neural network to solve this problem.It takes model parameters of each layer as input, learns parameterredundancies, and generates the dropout probabilities accordingly.Since there are interconnections of parameters among dierentlayers, we design the compressor neural network to be a recurrentneural network that can globally share the redundancy informationand generate dropout probabilities layer by layer.

    e compressor neural network is optimized jointly with the orig-inal neural network to be compressed through a compressor-criticframework that tries to minimize the loss function of the originalsensing applicaiton. e compressor-critic framework emulates theidea of the well-known actor-critic algorithm from reinforcementlearning [28], optimizing two networks in an iterative manner.

    We evaluate the DeepIoT framework on the Intel Edison comput-ing platform [1], which Intel markets as an enabler platform for thecomputing elements of embedded things in IoT systems. We con-duct two sets of experiments. e rst set consists of three tasks thatenable embedded systems to interact with humans with basic modal-ities, including handwrien text, vision, and speech, demonstratingsuperior accuracy of our produced neural networks, compared to

    others of similar size. e second set provides two examples ofapplying compressed neural networks to solving human-centriccontext sensing tasks; namely, human activity recognition and useridentication, in a resource-constrained scenario.

    We compare DeepIoT with other state-of-the-art magnitude-based [22] and sparse-coding-based [4] neural network compressionmethods. e resource consumption of resulting models on the IntelEdison module and the nal performance of sensing applications areestimated for all


View more >