An Interactive PCA visualizer

Singular Value Decomposition: Warding off curses

Statistics deals with finding order out of chaos. Sometimes, estimating variables is easy Oftentimes we find ourselves with more data than we know what to do with it. With more variables, more information. However, with more variables, there is also exponentially more complexity. This is called the CURSE OF DIMENSIONALITY. In order to avoid this fate, we use techniques for dimensionality reduction. That is, reducing the amount of variables down to just the most important ones. The techniques for achieving this fall under the umbrella of what is termed Spectral Theory. Spooky, but not so cursed.

A three-dimensional example

In this example, we randomly sample from a multivariate (many-variable) normal (continuous, linear) distribution. In order to not just get random noise, I added the option to input a covariance matrix. This means that each row represents how each variable depends on the other two variables. For each sample point, three independent uniformly-distributed variables from -1 to 1 are first put through the Mersenne Twister algorithm to generate normally-distributed samples, and then correlated via the Cholesky decomposition of the covariance matrix to form a linear multivariate distribution. That is to say, the three-dimensional sample points are stretched and rotated according to whatever you put into the matrix. I made this app using browser javascript, but it’s pretty computationally efficient. Feel free to give it a go! You can left-click and drag to pan around the axis center, and right-click drag to move the camera’s center.