Table of Contents
Deterministic Annealing for Clustering, Compression,
Classification, Regression, and Speech Recognition
Acknowledgement
References
Organization of this talk
I. Data/Space Partitioning
Data/Space Partitioning
Clustering
Classification
Solution: Bayesian Classifier
Functional Regression
Data Clustering
Thus,
Parametric Design
Example
Parametric Form
and in regression …
Parametric form:
Clustering
Standard approaches: GLA, K-means, etc.
The DA approach
Hard Partitioning
Random Partitioning
Best random partition
Problem
Analogy to Statistical Physics
Optimum at given T
Optimum at given T
DA Algorithm
Notes
More on the physical analogy
More on the physical analogy
More on the physical analogy
More on the physical analogy
Codebook Evolution
Codebook Evolution
Codebook Evolution
Codebook Evolution
Codebook Evolution
Codebook Evolution
Comparison with GLA
II. Extensions to Include Structural Constraints
DA with structured partition
Problem statement
Supervised learning
Unsupervised learning
Partition in canonical form
Partition rule
Randomized partition rule
Entropy constraint
Entropy constrained formulation
Effect of Constraint
Problem definition
Problem definition
Some Tested Applications
III. Design of HMM classifiers for speech recognition
Speech recognition
Hidden Markov Model (HMM)
Classification using HMMs
Most likely state sequence discriminant
Competition between paths
Classifier Design Problem
Maximum Likelihood Design
Direct Optimization
Descent methods for direct optimization
Deterministic Annealing
The Deterministic Annealing approach
Regular rule
Randomized rule
The DA approach
Random Decision
New cost
DA Algorithm
Notes
Experiment
Test set results for synthetic datasets
Speech Recognition Results
Experiment
Comparison of Error Rates (%)
Performance on the test set
Test set results
Continuous HMM Results
Continuous HMM Results
Experiment Results
Current Work - DA for SR
DA Applications So Far..
Conclusions |