learning a Nonlinear Embedding by Preserving Class Neighborhood Structure: Difference between revisions
No edit summary |
No edit summary |
||
Line 3: | Line 3: | ||
feature space in which K-nearest neighbour classification performs well. As the performance of algorithms like K-nearest neighbours (KNN) that are based on computing distances, the main objective of the proposed algorithm is to learn a good similarity measure that can provide insight into how high-dimensional data is organized. The nonlinear transformation is learned by pre-training and fine-tuning a multilayer neural network. The authors also show how to enhance the performance of non-linear transformation further using unlabeled data. Experimental results on a widely used version of the MNIST handwritten digit recognition task show that proposed algorithm achieves a much lower error rate than SVM or standard backpropagation. | feature space in which K-nearest neighbour classification performs well. As the performance of algorithms like K-nearest neighbours (KNN) that are based on computing distances, the main objective of the proposed algorithm is to learn a good similarity measure that can provide insight into how high-dimensional data is organized. The nonlinear transformation is learned by pre-training and fine-tuning a multilayer neural network. The authors also show how to enhance the performance of non-linear transformation further using unlabeled data. Experimental results on a widely used version of the MNIST handwritten digit recognition task show that proposed algorithm achieves a much lower error rate than SVM or standard backpropagation. | ||
==References | =Related work= | ||
== Neighborhood Component Analysis == | |||
=Nonlinear NCA= | |||
== Pre-training step == | |||
== Fine-tuning == | |||
=Regularized Nonlinear NCA= | |||
==Splitting codes into class-relevant and class-irrelevant parts== | |||
=Experiments= | |||
=References= | |||
<references/> | <references/> |
Revision as of 22:17, 30 June 2009
Introduction
The paper <ref>Salakhutdinov, R., & Hinton, G. E. (2007). Learning a nonlinear embedding by preserving class neighbourhood structure. AI and Statistics.</ref> presented here describes a method to learn a nonlinear transformation from the input space to a low-dimensional feature space in which K-nearest neighbour classification performs well. As the performance of algorithms like K-nearest neighbours (KNN) that are based on computing distances, the main objective of the proposed algorithm is to learn a good similarity measure that can provide insight into how high-dimensional data is organized. The nonlinear transformation is learned by pre-training and fine-tuning a multilayer neural network. The authors also show how to enhance the performance of non-linear transformation further using unlabeled data. Experimental results on a widely used version of the MNIST handwritten digit recognition task show that proposed algorithm achieves a much lower error rate than SVM or standard backpropagation.
Related work
Neighborhood Component Analysis
Nonlinear NCA
Pre-training step
Fine-tuning
Regularized Nonlinear NCA
Splitting codes into class-relevant and class-irrelevant parts
Experiments
References
<references/>