PCA and FA II

As the book by watsham tells us, PCA has two objectives:

• to reduce the dimensionality
• to interpret data.

Now I got confused with the reducing the dimensionality. As far as i know, it can only reduce linear combinations as non significant proportions of variability are eliminated. So I looked it up in myriad sources and found how can it actually reduce the dimensions.

This is how I got enlightened:

Let's say I have 100 stocks returns in an index and wanted to reduce or squeeze it into two or three variables. Since I have 100 stock returns, then I have also 100 principal components. The first of which (who has the highest eigenvalue) accounts for the highest variance and the 100th principal component accounts for the least as it is already ranked.  Note that what PCA do is it makes the principal components uncorrelated to each other. That means that even if the returns (input) itself is highly correlated as assumed, the principal components(output) which explains the variance are not.

There are two ways to do PCA. First is the eigenvalue decomposition (in the previous posts) and the other is  the singular value decomposition (still have to to research on this). Once we figure out the PC's that explain the majority of the variance, those PC's can serve as dependent variables in a regression. 

How is it different to factor analysis?

Even though PCAs appears to have "mathematically intuition", it is somehow lacks to have the "economic intuition". As I have understood, the PCA is only a subset of FA. FA group together variables according to its variability and in assumption that those variables are somewhat related or show characteristics that are the same.

Sources:
CAPM vs APT:
http://www.r-bloggers.com/principal-component-analysis-use-extended-to-financial-economics-part-2/

http://programming-r-pro-bro.blogspot.com/2011/10/principal-component-analysis-use.html