visualizing Data using t-SNE: Difference between revisions

From statwiki
Jump to navigation Jump to search
Line 3: Line 3:


=Stochastic Neighbor Embedding=
=Stochastic Neighbor Embedding=
The basic algorithm of SNE is showed as the follows.
<br> <center> <math> \mathbf  p_{j|i} = \frac{\exp(-||x_i-x_j ||^2/ 2\sigma_i ^2  )}{\sum_{k \neq i} \exp(-||x_i-x_k ||^2/ 2\sigma_i ^2 ) }</math> </center>
<br> <center> <math> q_{j|i} = \frac{\exp(-||y_i-y_j ||^2)}{\sum_{k \neq i} \exp(-||y_i-y_k ||^2) }</math> </center>
<br> <center> <math> C = \sum_{i} KL(P_i||Q_i) =\sum_{i}\sum_{j \neq i} p_{j|i} \log \frac{p_{j|i}}{q_{j|i}}</math> </center>


=t-Distributed Stochastic Neighbor Embedding=
=t-Distributed Stochastic Neighbor Embedding=

Revision as of 19:02, 12 July 2009

Introduction

The paper <ref>Laurens van der Maaten, and Geoffrey Hinton, 2008. Visualizing Data using t-SNE.</ref> introduced a new nonlinear dimensionally reduction technique that "embeds" high-dimensional data into low-dimensional space. This technique is a variation of the Stochastic Neighbor embedding (SNE) that was proposed by Hinton and Roweis in 2002 <ref>G.E. Hinton and S.T. Roweis, 2002. Stochastic Neighbor embedding.</ref>, where the high-dimensional Euclidean distances between datapoints are converted into the conditional probability to describe their similarities. t-SNE, based on the same idea, is aimed to be easier for optimization and to solve the "crowding problem". In addition, the author showed that t-SNE can be applied to large data sets as well, by using random walks on neighborhood graphs. The performance of t-SNE is demonstrated on a wide variety of data sets and compared with many other visualization techniques.

Stochastic Neighbor Embedding

The basic algorithm of SNE is showed as the follows.


[math]\displaystyle{ \mathbf p_{j|i} = \frac{\exp(-||x_i-x_j ||^2/ 2\sigma_i ^2 )}{\sum_{k \neq i} \exp(-||x_i-x_k ||^2/ 2\sigma_i ^2 ) } }[/math]



[math]\displaystyle{ q_{j|i} = \frac{\exp(-||y_i-y_j ||^2)}{\sum_{k \neq i} \exp(-||y_i-y_k ||^2) } }[/math]


[math]\displaystyle{ C = \sum_{i} KL(P_i||Q_i) =\sum_{i}\sum_{j \neq i} p_{j|i} \log \frac{p_{j|i}}{q_{j|i}} }[/math]

t-Distributed Stochastic Neighbor Embedding

Symmetric SNE

The Crowding Problem

Compensating for Mismatched Dimensionality by Mismatched Tails

Optimization Methods for t-SNE

Experiments with Different Data Sets

t-SNE for Large Data Sets

Weaknesses of t-SNE

Summary

References