Deterministic Annealing for Clustering, Compression, Classification, Regression, and Speech Recognition


Click here to start


Table of Contents

Deterministic Annealing for Clustering, Compression, Classification, Regression, and Speech Recognition

Acknowledgement

References

Organization of this talk

I. Data/Space Partitioning

Data/Space Partitioning

Clustering

Classification

Solution: Bayesian Classifier

Functional Regression

Data Clustering

Thus,

Parametric Design

Example

Parametric Form

and in regression …

Parametric form:

Clustering

Standard approaches: GLA, K-means, etc.

The DA approach

Hard Partitioning

Random Partitioning

Best random partition

Problem

Analogy to Statistical Physics

Optimum at given T

Optimum at given T

DA Algorithm

Notes

More on the physical analogy

More on the physical analogy

More on the physical analogy

More on the physical analogy

Codebook Evolution

Codebook Evolution

Codebook Evolution

Codebook Evolution

Codebook Evolution

Codebook Evolution

Comparison with GLA

II. Extensions to Include Structural Constraints

DA with structured partition

Problem statement

Supervised learning

Unsupervised learning

Partition in canonical form

Partition rule

Randomized partition rule

Entropy constraint

Entropy constrained formulation

Effect of Constraint

Problem definition

Problem definition

Some Tested Applications

III. Design of HMM classifiers for speech recognition

Speech recognition

Hidden Markov Model (HMM)

Classification using HMMs

Most likely state sequence discriminant

Competition between paths

Classifier Design Problem

Maximum Likelihood Design

Direct Optimization

Descent methods for direct optimization

Deterministic Annealing

The Deterministic Annealing approach

Regular rule

Randomized rule

The DA approach

Random Decision

New cost

DA Algorithm

Notes

Experiment

Test set results for synthetic datasets

Speech Recognition Results

Experiment

Comparison of Error Rates (%)

Performance on the test set

Test set results

Continuous HMM Results

Continuous HMM Results

Experiment Results

Current Work - DA for SR

DA Applications So Far..

Conclusions

Author: Dr. Kenneth Rose

Email: rose@ece.ucsb.edu

Home Page: http://www.ece.ucsb.edu/Faculty/Rose/default.html