Incremental Class Learning Approach and Its Application to Handwritten Digit Recognition

TitleIncremental Class Learning Approach and Its Application to Handwritten Digit Recognition
Publication TypeTechnical Report
Year of Publication1998
AuthorsMańdziuk, J., & Shastri L.
Other Numbers1136
Keywordscatastrophic interference problem, Handwritten Digit Recognition, Incremental Class Learning, neural network, pattern recognition, spatio-temporal representation, supervised learning

Incremental Class Learning (ICL) provides a feasible framework for the development of scalable learning systems. Instead of learning a complex problem at once, ICL focuses on learning subproblems incrementally, one at a time - using the results of prior learning for subsequent learning - and then combining the solutions in an appropriate manner. With respect to multi-class classification problems, the ICL approach presented in this paper can be summarized as follows. Initially the system focuses on one category. After it learns this category, it tries to identify a compact subset of features (nodes) in the hidden layers, that are crucial for the recognition of this category. The system then freezes these crucial nodes (features) by fixing their incoming weights. As a result, these features cannot be obliterated in subsequent learning. These frozen features are available during subsequent learning and can serve as parts of weight structures build to recognize other categories. As more categories are learned, the set of features gradually stabilizes and learning a new category requires less effort. Eventually, learning a new category may only involve combining existing features in an appropriate manner. The approach promotes the sharing of learned features among a number of categories and also alleviates the well-known catastrophic interference problem. We present results of applying the ICL approach to the Handwritten Digit Recognition problem, based on a spatio-temporal representation of patterns.

Bibliographic Notes

ICSI Technical Report TR-98-015

Abbreviated Authors

J. Mandziuk and L. Shastri

ICSI Publication Type

Technical Report