ICE-Pick: Iterative Cost-Efficient Pruning for DNNs

Hu, W., Gibson, P. and Cano, J. (2023) ICE-Pick: Iterative Cost-Efficient Pruning for DNNs. ICML 2023 Workshop on Neural Compression, Honolulu, Hawaii, USA, 28-29 July 2023.

[img] Text
303684.pdf - Published Version



Pruning is one of the main compression methods for Deep Neural Networks (DNNs), where less relevant parameters are removed from a DNN model to reduce its memory footprint. To get better final accuracy, pruning is often performed iteratively with increasing amounts of parameters being removed in each step, and fine-tuning (i.e., additional training epochs) being applied to the remaining parameters. However, this process can be very time-consuming, since the finetuning process is applied after every pruning step and calculates gradients for the whole model. Motivated by these overheads, in this paper we propose ICE-Pick, a novel threshold-guided finetuning method which freezes less sensitive layers and leverages a custom pruning-aware learning rate scheduler. We evaluate ICE-Pick using ResNet-110, ResNet-152, and MobileNetV2 (all defined for CIFAR-10), and show that it can save up to 87.6% of the pruning time while maintaining accuracy.

Item Type:Conference or Workshop Item
Glasgow Author(s) Enlighten ID:Cano Reyes, Dr Jose and Gibson, Mr Perry
Authors: Hu, W., Gibson, P., and Cano, J.
College/School:College of Science and Engineering > School of Computing Science
Copyright Holders:Copyright 2023 by the author(s).
Publisher Policy:Reproduced with the permission of the authors
Related URLs:

University Staff: Request a correction | Enlighten Editors: Update this record