Publication Details
Title: Adaptive Parameter Pruning in Neural Networks
Author: L. Prechelt
Group: ICSI Technical Reports
Date: March 1995
PDF: ftp://ftp.icsi.berkeley.edu/pub/techreports/1995/tr-95-009.pdf
Overview:
Neural network pruning methods on the level of individual network parameters (e.g. connection weights) can improve generalization. An open problem in the pruning methods known today (OBD, OBS, autoprune, epsiprune) is the selection of the number of parameters to be removed in each pruning step (pruning strength). This paper presents a pruning method "lprune" that automatically adapts the pruning strength to the evolution of weights and loss of generalization during training. The method requires no algorithm parameter adjustment by the user. The results of extensive experimentation indicate that lprune is often superior to autoprune (which is superior to OBD) on diagnosis tasks unless severe pruning early in the training process is required. Results of statistical significance tests comparing autoprune to the new method lprune as well as to backpropagation with early stopping are given for 14 different problems.
Bibliographic Information:
ICSI Technical Report TR-95-009
Bibliographic Reference:
L. Prechelt. Adaptive Parameter Pruning in Neural Networks. ICSI Technical Report TR-95-009, March 1995
Author: L. Prechelt
Group: ICSI Technical Reports
Date: March 1995
PDF: ftp://ftp.icsi.berkeley.edu/pub/techreports/1995/tr-95-009.pdf
Overview:
Neural network pruning methods on the level of individual network parameters (e.g. connection weights) can improve generalization. An open problem in the pruning methods known today (OBD, OBS, autoprune, epsiprune) is the selection of the number of parameters to be removed in each pruning step (pruning strength). This paper presents a pruning method "lprune" that automatically adapts the pruning strength to the evolution of weights and loss of generalization during training. The method requires no algorithm parameter adjustment by the user. The results of extensive experimentation indicate that lprune is often superior to autoprune (which is superior to OBD) on diagnosis tasks unless severe pruning early in the training process is required. Results of statistical significance tests comparing autoprune to the new method lprune as well as to backpropagation with early stopping are given for 14 different problems.
Bibliographic Information:
ICSI Technical Report TR-95-009
Bibliographic Reference:
L. Prechelt. Adaptive Parameter Pruning in Neural Networks. ICSI Technical Report TR-95-009, March 1995