Please use this identifier to cite or link to this item:
https://doi.org/10.1109/ACCESS.2019.2957203
DC Field | Value | |
---|---|---|
dc.title | Pruning Blocks for CNN Compression and Acceleration via Online Ensemble Distillation | |
dc.contributor.author | Wang, Z. | |
dc.contributor.author | Lin, S. | |
dc.contributor.author | Xie, J. | |
dc.contributor.author | Lin, Y. | |
dc.date.accessioned | 2022-01-11T06:20:21Z | |
dc.date.available | 2022-01-11T06:20:21Z | |
dc.date.issued | 2019 | |
dc.identifier.citation | Wang, Z., Lin, S., Xie, J., Lin, Y. (2019). Pruning Blocks for CNN Compression and Acceleration via Online Ensemble Distillation. IEEE Access 7 : 175703-175716. ScholarBank@NUS Repository. https://doi.org/10.1109/ACCESS.2019.2957203 | |
dc.identifier.issn | 21693536 | |
dc.identifier.uri | https://scholarbank.nus.edu.sg/handle/10635/213735 | |
dc.description.abstract | In this paper, we propose an online ensemble distillation (OED) method to automatically prune blocks/layers of a target network by transferring the knowledge from a strong teacher in an end-to-end manner. To accomplish this, we first introduce a soft mask to scale the output of each block in the target network and enforce the sparsity of the mask by sparsity regularization. Then, a strong teacher network is constructed online by replicating the same target networks and ensembling the discriminative features from each target as its new features. Cooperative learning between multiple target networks and the teacher network is further conducted in a closed-loop form, which improves their performance. To solve the optimization problem in an end-to-end manner, we employ the fast iterative shrinkage-thresholding algorithm to fast and reliably remove the redundant blocks, in which the corresponding soft masks are equal to zero. Compared to other structured pruning methods with iterative fine-tuning, the proposed OED is trained more efficiently in one training cycle. Extensive experiments demonstrate the effectiveness of OED, which can not only simultaneously compress and accelerate a variety of CNN architectures but also enhance the robustness of the pruned networks. © 2013 IEEE. | |
dc.publisher | Institute of Electrical and Electronics Engineers Inc. | |
dc.rights | Attribution-NonCommercial-NoDerivatives 4.0 International | |
dc.rights.uri | https://creativecommons.org/licenses/by-nc-nd/4.0/ | |
dc.source | Scopus OA2019 | |
dc.subject | Fast iterative shrinkage-thresholding algorithm | |
dc.subject | model compression and acceleration | |
dc.subject | network pruning | |
dc.subject | online ensemble distillation | |
dc.type | Article | |
dc.contributor.department | DEPT OF COMPUTER SCIENCE | |
dc.description.doi | 10.1109/ACCESS.2019.2957203 | |
dc.description.sourcetitle | IEEE Access | |
dc.description.volume | 7 | |
dc.description.page | 175703-175716 | |
Appears in Collections: | Staff Publications Elements |
Show simple item record
Files in This Item:
File | Description | Size | Format | Access Settings | Version | |
---|---|---|---|---|---|---|
10_1109_ACCESS_2019_2957203.pdf | 7.51 MB | Adobe PDF | OPEN | None | View/Download |
This item is licensed under a Creative Commons License