Smart Pruning of Deep Neural Networks Using Curve Fitting and Evolution of Weights

Ashhadul Islam; Samir Brahim Belhaouari

doi:10.1007/978-3-031-25891-6_6

Smart Pruning of Deep Neural Networks Using Curve Fitting and Evolution of Weights

Ashhadul Islam^*, Samir Brahim Belhaouari

^*Corresponding author for this work

CSE Information & Computing Technology

Hamad bin Khalifa University

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

1 Citation (Scopus)

Abstract

Compression of the deep neural networks is a critical problem area when it comes to enhancing the capability of embedded devices. As deep neural networks are space and compute-intensive, they are generally unsuitable for use in edge devices and thereby lose their ubiquity. This paper discusses novel methods of neural network pruning, making them lighter, faster, and immune to noise and over-fitting without compromising the accuracy of the models. It poses two questions about the accepted methods of pruning and proffers two new strategies - evolution of weights and smart pruning to compress the deep neural networks better. These methods are then compared with the standard pruning mechanism on benchmark data sets to establish their efficiency. The code is made available online for public use.

Original language	English
Title of host publication	Machine Learning, Optimization, And Data Science, Lod 2022, Pt Ii
Editors	G Nicosia, V Ojha, E LaMalfa, G LaMalfa, P Pardalos, G DiFatta, G Giuffrida, R Umeton
Publisher	Springer Science and Business Media Deutschland GmbH
Pages	62-76
Number of pages	15
Volume	13811
ISBN (Electronic)	978-3-031-25891-6
ISBN (Print)	9783031258909
DOIs	https://doi.org/10.1007/978-3-031-25891-6_6
Publication status	Published - 2023
Event	8th International Conference on Machine Learning, Optimization, and Data Science, LOD 2022, held in conjunction with the 2nd Advanced Course and Symposium on Artificial Intelligence and Neuroscience, ACAIN 2022 - Certosa di Pontignano, Italy Duration: 18 Sept 2022 → 22 Sept 2022

Publication series

Name	Lecture Notes In Computer Science

Conference

Conference	8th International Conference on Machine Learning, Optimization, and Data Science, LOD 2022, held in conjunction with the 2nd Advanced Course and Symposium on Artificial Intelligence and Neuroscience, ACAIN 2022
Country/Territory	Italy
City	Certosa di Pontignano
Period	18/09/22 → 22/09/22

Keywords

Deep neural networks
Explainability
Forward optimization
Pruning
Weight manipulation

Access to Document

10.1007/978-3-031-25891-6_6

Cite this

Islam, A., & Belhaouari, S. B. (2023). Smart Pruning of Deep Neural Networks Using Curve Fitting and Evolution of Weights. In G. Nicosia, V. Ojha, E. LaMalfa, G. LaMalfa, P. Pardalos, G. DiFatta, G. Giuffrida, & R. Umeton (Eds.), Machine Learning, Optimization, And Data Science, Lod 2022, Pt Ii (Vol. 13811, pp. 62-76). (Lecture Notes In Computer Science). Springer Science and Business Media Deutschland GmbH. https://doi.org/10.1007/978-3-031-25891-6_6

Islam, Ashhadul ; Belhaouari, Samir Brahim. / Smart Pruning of Deep Neural Networks Using Curve Fitting and Evolution of Weights. Machine Learning, Optimization, And Data Science, Lod 2022, Pt Ii. editor / G Nicosia ; V Ojha ; E LaMalfa ; G LaMalfa ; P Pardalos ; G DiFatta ; G Giuffrida ; R Umeton. Vol. 13811 Springer Science and Business Media Deutschland GmbH, 2023. pp. 62-76 (Lecture Notes In Computer Science).

@inproceedings{2effee718c56422dad1a8c448e21ba40,

title = "Smart Pruning of Deep Neural Networks Using Curve Fitting and Evolution of Weights",

abstract = "Compression of the deep neural networks is a critical problem area when it comes to enhancing the capability of embedded devices. As deep neural networks are space and compute-intensive, they are generally unsuitable for use in edge devices and thereby lose their ubiquity. This paper discusses novel methods of neural network pruning, making them lighter, faster, and immune to noise and over-fitting without compromising the accuracy of the models. It poses two questions about the accepted methods of pruning and proffers two new strategies - evolution of weights and smart pruning to compress the deep neural networks better. These methods are then compared with the standard pruning mechanism on benchmark data sets to establish their efficiency. The code is made available online for public use.",

keywords = "Deep neural networks, Explainability, Forward optimization, Pruning, Weight manipulation",

author = "Ashhadul Islam and Belhaouari, {Samir Brahim}",

note = "Publisher Copyright: {\textcopyright} 2023, The Author(s), under exclusive license to Springer Nature Switzerland AG.; 8th International Conference on Machine Learning, Optimization, and Data Science, LOD 2022, held in conjunction with the 2nd Advanced Course and Symposium on Artificial Intelligence and Neuroscience, ACAIN 2022 ; Conference date: 18-09-2022 Through 22-09-2022",

year = "2023",

doi = "10.1007/978-3-031-25891-6_6",

language = "English",

isbn = "9783031258909",

volume = "13811",

series = "Lecture Notes In Computer Science",

publisher = "Springer Science and Business Media Deutschland GmbH",

pages = "62--76",

editor = "G Nicosia and V Ojha and E LaMalfa and G LaMalfa and P Pardalos and G DiFatta and G Giuffrida and R Umeton",

booktitle = "Machine Learning, Optimization, And Data Science, Lod 2022, Pt Ii",

address = "Germany",

}

Islam, A & Belhaouari, SB 2023, Smart Pruning of Deep Neural Networks Using Curve Fitting and Evolution of Weights. in G Nicosia, V Ojha, E LaMalfa, G LaMalfa, P Pardalos, G DiFatta, G Giuffrida & R Umeton (eds), Machine Learning, Optimization, And Data Science, Lod 2022, Pt Ii. vol. 13811, Lecture Notes In Computer Science, Springer Science and Business Media Deutschland GmbH, pp. 62-76, 8th International Conference on Machine Learning, Optimization, and Data Science, LOD 2022, held in conjunction with the 2nd Advanced Course and Symposium on Artificial Intelligence and Neuroscience, ACAIN 2022, Certosa di Pontignano, Italy, 18/09/22. https://doi.org/10.1007/978-3-031-25891-6_6

Smart Pruning of Deep Neural Networks Using Curve Fitting and Evolution of Weights. / Islam, Ashhadul; Belhaouari, Samir Brahim.
Machine Learning, Optimization, And Data Science, Lod 2022, Pt Ii. ed. / G Nicosia; V Ojha; E LaMalfa; G LaMalfa; P Pardalos; G DiFatta; G Giuffrida; R Umeton. Vol. 13811 Springer Science and Business Media Deutschland GmbH, 2023. p. 62-76 (Lecture Notes In Computer Science).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

TY - GEN

T1 - Smart Pruning of Deep Neural Networks Using Curve Fitting and Evolution of Weights

AU - Islam, Ashhadul

AU - Belhaouari, Samir Brahim

PY - 2023

Y1 - 2023

N2 - Compression of the deep neural networks is a critical problem area when it comes to enhancing the capability of embedded devices. As deep neural networks are space and compute-intensive, they are generally unsuitable for use in edge devices and thereby lose their ubiquity. This paper discusses novel methods of neural network pruning, making them lighter, faster, and immune to noise and over-fitting without compromising the accuracy of the models. It poses two questions about the accepted methods of pruning and proffers two new strategies - evolution of weights and smart pruning to compress the deep neural networks better. These methods are then compared with the standard pruning mechanism on benchmark data sets to establish their efficiency. The code is made available online for public use.

AB - Compression of the deep neural networks is a critical problem area when it comes to enhancing the capability of embedded devices. As deep neural networks are space and compute-intensive, they are generally unsuitable for use in edge devices and thereby lose their ubiquity. This paper discusses novel methods of neural network pruning, making them lighter, faster, and immune to noise and over-fitting without compromising the accuracy of the models. It poses two questions about the accepted methods of pruning and proffers two new strategies - evolution of weights and smart pruning to compress the deep neural networks better. These methods are then compared with the standard pruning mechanism on benchmark data sets to establish their efficiency. The code is made available online for public use.

KW - Deep neural networks

KW - Explainability

KW - Forward optimization

KW - Pruning

KW - Weight manipulation

UR - http://www.scopus.com/inward/record.url?scp=85151056851&partnerID=8YFLogxK

U2 - 10.1007/978-3-031-25891-6_6

DO - 10.1007/978-3-031-25891-6_6

M3 - Conference contribution

AN - SCOPUS:85151056851

SN - 9783031258909

VL - 13811

T3 - Lecture Notes In Computer Science

SP - 62

EP - 76

BT - Machine Learning, Optimization, And Data Science, Lod 2022, Pt Ii

A2 - Nicosia, G

A2 - Ojha, V

A2 - LaMalfa, E

A2 - LaMalfa, G

A2 - Pardalos, P

A2 - DiFatta, G

A2 - Giuffrida, G

A2 - Umeton, R

PB - Springer Science and Business Media Deutschland GmbH

T2 - 8th International Conference on Machine Learning, Optimization, and Data Science, LOD 2022, held in conjunction with the 2nd Advanced Course and Symposium on Artificial Intelligence and Neuroscience, ACAIN 2022

Y2 - 18 September 2022 through 22 September 2022

ER -

Islam A, Belhaouari SB. Smart Pruning of Deep Neural Networks Using Curve Fitting and Evolution of Weights. In Nicosia G, Ojha V, LaMalfa E, LaMalfa G, Pardalos P, DiFatta G, Giuffrida G, Umeton R, editors, Machine Learning, Optimization, And Data Science, Lod 2022, Pt Ii. Vol. 13811. Springer Science and Business Media Deutschland GmbH. 2023. p. 62-76. (Lecture Notes In Computer Science). doi: 10.1007/978-3-031-25891-6_6