Publication Details
Title: VC Dimension of Sigmoidal and General Pfaffian Neural Networks
Author: M. Karpinski and A. Macintyre
Group: ICSI Technical Reports
Date: November 1995
PDF: ftp://ftp.icsi.berkeley.edu/pub/techreports/1995/tr-95-065.pdf
Overview:
We introduce a new method for proving explicit upper bounds on the VC Dimension of general functional basis networks, and prove as an application, for the first time, that the VC Dimension of analog neural networks with the sigmoidal activation function sigma(y)=1/1+e^{-y} is bounded by a quadratic polynomial O((lm)^2) in both the number l of programmable parameters, and the number m of nodes. The proof method of this paper generalizes to much wider class of Pfaffian activation functions and formulas, and gives also for the first time polynomial bounds on their VC Dimension. We present also some other applications of our method. Keywords: VC Dimension, Pfaffian Activation Functions and Formulas, Neural Networks, Sparse Networks, Boolean Computation
Bibliographic Information:
ICSI Technical Report TR-95-065
Bibliographic Reference:
M. Karpinski and A. Macintyre. VC Dimension of Sigmoidal and General Pfaffian Neural Networks. ICSI Technical Report TR-95-065, November 1995
Author: M. Karpinski and A. Macintyre
Group: ICSI Technical Reports
Date: November 1995
PDF: ftp://ftp.icsi.berkeley.edu/pub/techreports/1995/tr-95-065.pdf
Overview:
We introduce a new method for proving explicit upper bounds on the VC Dimension of general functional basis networks, and prove as an application, for the first time, that the VC Dimension of analog neural networks with the sigmoidal activation function sigma(y)=1/1+e^{-y} is bounded by a quadratic polynomial O((lm)^2) in both the number l of programmable parameters, and the number m of nodes. The proof method of this paper generalizes to much wider class of Pfaffian activation functions and formulas, and gives also for the first time polynomial bounds on their VC Dimension. We present also some other applications of our method. Keywords: VC Dimension, Pfaffian Activation Functions and Formulas, Neural Networks, Sparse Networks, Boolean Computation
Bibliographic Information:
ICSI Technical Report TR-95-065
Bibliographic Reference:
M. Karpinski and A. Macintyre. VC Dimension of Sigmoidal and General Pfaffian Neural Networks. ICSI Technical Report TR-95-065, November 1995