Soft-to-Hard Model Transition in Clustering: A Review

TitleSoft-to-Hard Model Transition in Clustering: A Review
Publication TypeTechnical Report
Year of Publication1999
AuthorsBaraldi, A., & Schenato L..
Other Numbers1166
Keywordsquantization error, soft and hard competitive clustering algorithms, unsupervised learning

Clustering analysis often employs unsupervised learning techniques originally developed for vector quantization. In this framework, a frequent goal of clustering systems is to minimize the {it quantization error}, which is affected by many local minima. To avoid confinement of reference vectors to local minima of the quantization error and to avoid formation of dead units, hard c-means clustering algorithms are traditionally adapted by replacing their hard competitive strategy with a soft adaptation rule, where the degree of overlap between receptive fields is proportional to a monotonically decreasing scale (temperature) parameter. By starting at a high temperature, which is carefully lowered to zero, a soft-to-hard competitive clustering model transition is pursued, such that local minima of the quantization error are expected to emerge slowly, thereby preventing the set of reference vectors from being trapped in suboptimal states. A review of the hard c-means, Maximum-Entropy, Fuzzy Learning Vector Quantization (FLVQ), Neural Gas (NG), Self-Organizing Map (SOM) and a mixture of Gaussians method is provided, relationships between these methods are highlighted and a possible criterion for discriminating between different soft-to-hard competitive clustering model transitions is suggested.

Bibliographic Notes

ICSI Technical Report TR-99-010

Abbreviated Authors

A. Baraldi and L. Schenato

ICSI Research Group


ICSI Publication Type

Technical Report