Publication Details
Title: Soft-to-Hard Model Transition in Clustering: A Review
Author: A. Baraldi and L. Schenato
Group: ICSI Technical Reports
Date: September 1999
PDF: ftp://ftp.icsi.berkeley.edu/pub/techreports/1999/tr-99-010.pdf
Overview:
Clustering analysis often employs unsupervised learning techniques originally developed for vector quantization. In this framework, a frequent goal of clustering systems is to minimize the {it quantization error}, which is affected by many local minima. To avoid confinement of reference vectors to local minima of the quantization error and to avoid formation of dead units, hard c-means clustering algorithms are traditionally adapted by replacing their hard competitive strategy with a soft adaptation rule, where the degree of overlap between receptive fields is proportional to a monotonically decreasing scale (temperature) parameter. By starting at a high temperature, which is carefully lowered to zero, a soft-to-hard competitive clustering model transition is pursued, such that local minima of the quantization error are expected to emerge slowly, thereby preventing the set of reference vectors from being trapped in suboptimal states. A review of the hard c-means, Maximum-Entropy, Fuzzy Learning Vector Quantization (FLVQ), Neural Gas (NG), Self-Organizing Map (SOM) and a mixture of Gaussians method is provided, relationships between these methods are highlighted and a possible criterion for discriminating between different soft-to-hard competitive clustering model transitions is suggested. Keywords: unsupervised learning, soft and hard competitive clustering algorithms, quantization error.
Bibliographic Information:
ICSI Technical Report TR-99-010
Bibliographic Reference:
A. Baraldi and L. Schenato. Soft-to-Hard Model Transition in Clustering: A Review. ICSI Technical Report TR-99-010, September 1999
Author: A. Baraldi and L. Schenato
Group: ICSI Technical Reports
Date: September 1999
PDF: ftp://ftp.icsi.berkeley.edu/pub/techreports/1999/tr-99-010.pdf
Overview:
Clustering analysis often employs unsupervised learning techniques originally developed for vector quantization. In this framework, a frequent goal of clustering systems is to minimize the {it quantization error}, which is affected by many local minima. To avoid confinement of reference vectors to local minima of the quantization error and to avoid formation of dead units, hard c-means clustering algorithms are traditionally adapted by replacing their hard competitive strategy with a soft adaptation rule, where the degree of overlap between receptive fields is proportional to a monotonically decreasing scale (temperature) parameter. By starting at a high temperature, which is carefully lowered to zero, a soft-to-hard competitive clustering model transition is pursued, such that local minima of the quantization error are expected to emerge slowly, thereby preventing the set of reference vectors from being trapped in suboptimal states. A review of the hard c-means, Maximum-Entropy, Fuzzy Learning Vector Quantization (FLVQ), Neural Gas (NG), Self-Organizing Map (SOM) and a mixture of Gaussians method is provided, relationships between these methods are highlighted and a possible criterion for discriminating between different soft-to-hard competitive clustering model transitions is suggested. Keywords: unsupervised learning, soft and hard competitive clustering algorithms, quantization error.
Bibliographic Information:
ICSI Technical Report TR-99-010
Bibliographic Reference:
A. Baraldi and L. Schenato. Soft-to-Hard Model Transition in Clustering: A Review. ICSI Technical Report TR-99-010, September 1999