Showing posts with label PCA and FA. Show all posts
Showing posts with label PCA and FA. Show all posts

Thursday, February 14, 2013

PCA and FA II

As the book by watsham tells us, PCA has two objectives:

  • to reduce the dimensionality
  • to interpret data.
Now I got confused with the reducing the dimensionality. As far as i know, it can only reduce linear combinations as non significant proportions of variability are eliminated. So I looked it up in myriad sources and found how can it actually reduce the dimensions.

This is how I got enlightened:

Let's say I have 100 stocks returns in an index and wanted to reduce or squeeze it into two or three variables. Since I have 100 stock returns, then I have also 100 principal components. The first of which (who has the highest eigenvalue) accounts for the highest variance and the 100th principal component accounts for the least as it is already ranked.  Note that what PCA do is it makes the principal components uncorrelated to each other. That means that even if the returns (input) itself is highly correlated as assumed, the principal components(output) which explains the variance are not.

There are two ways to do PCA. First is the eigenvalue decomposition (in the previous posts) and the other is  the singular value decomposition (still have to to research on this). Once we figure out the PC's that explain the majority of the variance, those PC's can serve as dependent variables in a regression. 

How is it different to factor analysis?

Even though PCAs appears to have "mathematically intuition", it is somehow lacks to have the "economic intuition". As I have understood, the PCA is only a subset of FA. FA group together variables according to its variability and in assumption that those variables are somewhat related or show characteristics that are the same.

Sources:
CAPM vs APT:
http://www.r-bloggers.com/principal-component-analysis-use-extended-to-financial-economics-part-2/

http://programming-r-pro-bro.blogspot.com/2011/10/principal-component-analysis-use.html

PCA and FA I

Principal Components Analysis 
    - answers the question " what are the major contributors to the overall variability?"
    - so we look at the "dependent" variables and analyze them


Sample Application:

  • Age Groups as variable - who among the age group contributes more. 
  • What moves the bond return the most? 
    • First, we make a lot of guesses on the x's variables such as:
      • tenor
      • coupon rate
      • yield
      • credit rating
      • duration
      • demand for the bond
      • inflation expectation
      • supply
      • money supply
      • political stability
      • GDP expectation
    • Next, we eliminate the some of the variables based on redundancy, data, etc.
      • credit ratings are all risk free (means 0 variance), so we eliminate it.
      • demand and supply are related. It can be considered as one. 
      • political stability is not measurable.
    • Then, we determine the data for the other variables as they are measurable
      • tenor
      • coupon rate
      • yield
      • duration
      • demand and supply for the bond - bid ask spread
      • inflation expectation - BSP futures curve or consensus estimate for GDP
      • money supply - entry growth BSP
      • GDP expectation 
    • we now can apply PCA to the variables
Applying PCA to the x variables
  • From the variance-covariance matrix, we get the correlation matrix. It is so because the data needs to be normalized first and we are able to achieve it by dividing the data by the standard deviation. And correlation matrix is the standardized form of the variance-covariance matrix.
  • Then get eigenvectors and the eigenvalues.
    • Eigenvectors - produce the matrix of linear combinations 
    • Eigenvalues - its total sum is the total number of variables in the standardized matrix
    • each eigenvector (linear combination) has its own eigenvalue 
  • Interpretation of the eigenvectors and the eigenvalues
    • divide the each eigenvalue by its sum gives us the proportion that eigenvector accounts to the total variance. The higher the proportion the more significant its contribution to the total variability. 
    • Some says that getting eigenvalues greater than 1 are the ones that are significant. So we disregard the ones that are less than one since they are not significant.
  • As application in portfolio management, this gives the portfolio manager the opportunity to adjust the weights of the portfolio components so as to reduce the element of risk.
  Factor Analysis
  • main focus: summarization and data reduction
  • Given the variables, it answers the question "What drives the variables?"
  • this analysis is an ends by itself  or can be a part of a process
  • Stock = A Factor1 + B Factor2 + C Factor3, where the factors are the groups.
Example: 

Given stocks 1-10, (in the dummy stocks excel, and quant prof) we apply factor analysis. First we get the correlation matrix from group statistics, and then apply factor analysis with the correlation matrix.

From the unrotated loadings, we determine each group or factor by ranking as shown on the example below.


There are other ways to get the factor analysis. What is important is that we know how to analyze the data given.

This can be one of the individual projects. can also be ANOVA, test of hypothesis, etc.

PCA and FA application - factoranalysis.xlsx

Consider the linear combinations of 3 factors given below. We assumed that there are 3 factors affecting stockA, B and C.


For Problem1, we want to find the weights to invest in stock A,B and C, considering equal returns for the three stocks. For Problem2, we want to find the weights to invest in stock A,B and C, considering only Stock C returns. This is called one factor analysis. For Problem3, we want to find the weights to invest in stock A,B and C, considering only Stock A and half of Stock C returns. The problem is summarized on the table below.

 Solution:

To solve for the weights, we used the "matrix OLS" as a solution.
\[\hat{y} = \hat{x}\hat{b} \to \hat{b} = \hat{x}^{-1}\hat{y}\]
 In this case $\hat{y}$ is our expected returns, $\hat{b}$ is our weights and \hat{x} is the factors affecting the returns.

With problem 1, we applied multi-factor sensitivity as all of the factors and stocks are considered. After solving for the weights, we have to buy 47% of stock A, sell 77% of stock B and buy 145% of Stock C. Also it is found that we have to borrow 15% since the total sum of the weights is 115%.

Problem 2 only considered Stock C. When we do this, one factor sensitivity is used. As a result in finding for the weights, we have to sell 145% and 174% for stocks A and C respectively. (Can we sell what we don't have??) The total sum of the weights are 9% meaning it is not fully allocated.
 

Problem 3 considers a return for Stock A and half of Stock C only. This results to allocating 60%, 130% and -134% to the stocks A, B and C respectively. Total sum of the weights is 55% which pertains to the total allocation.



Sources: ppt class presentations, class lectures and quantprof VBA

PCA and FA Intro

Difference between PCA and FA to Others

Recall:

Multiple Regression (MR): relationship between an exogenous variable and many endogenous variable

Cointegration: endogenous variables are stationary (means the joint probability distribution does not shift through time and space, mean and variance remain constant throughout time and position)

Mean-Variance Analysis: measures the total collective variability of a group of variables, without specifically identifying which subgroups contribute to that variability

PCA and FA
  • compared to MR examines only the endogenous variables
  • compared to Cointegration it may not need to be stationary
  • compared to Mean-Variance Analysis, PCA identify and rank subgroups and their contribution to the total variability
  • Both uses variance-covariance matrix
Characteristics of Mutivariate Statistics
  • Volatility in the multivariate structure
  • Correlation or colinearity between variables
Principal Components Analysis

- volatility of multivariate structure is measured and analyzed
- total variability is measured by the sum of the eigenvalues (sum of the diagonals in the matrix)

Factor Analysis
- correlation between the variables of  a multivariate structure is analyzed

Sources: Quantitative Methods in Finance by Watsham and Parramore