Least Median of Squares Estimation by Optimization Heuristics with an Application to CAPM


Marianna Lyra1


Beta, LMS, CAPM, Differential Evolution, Threshold Accepting

Review Status

General review by COMISEF Wiki Admin, 09/2008


For estimating the parameters of models for financial market data at higher frequency, the use of robust techniques is of particular interest.
While only the most basic capital asset pricing model (CAPM) is considered, extensions to more refined models are straightforward. It is proposed to consider least median of squares (LMS) estimators in this context. Given the complexity of the objective function, the estimates are obtained by means of optimization heuristics. The performance of two heuristics is compared, namely differential evolution and threshold accepting. It is shown that these methods are well suited to obtain least median of squares estimators for real world problems such as the CAPM. Furthermore, it is analyzed to what extent parameter estimates and conditional forecasts based on the LMS differ from those obtained by OLS. The empirical analysis considers some stocks from the Dow Jones Industrial Average Index (DJIA) for different sample periods. Although estimation appears to be feasible using the heuristics proposed, the findings for the CAPM are rather mixed.

1 Introduction

Traditionally, the (CAPM) provides the method for estimating the risk-return equilibrium. The following single factor model provides
a linear relationship between the risk premium on individualsecurities relative to the risk premium on the market portfolio where, $r_{t}^s$ is the risk free rate of return at time $t $, $ r_{i,t}$ is the rate of return at time $t$ for asset $i $, $ r_{m,t}$ market rate of return at time $t$, $\alpha,\beta$ are the parameters of CAPM to be estimated and $\varepsilon_{i,t}$ residual at time $t$ for asset $i$.

\begin{align} r_{i,t}-r_{t}^s=\alpha+\beta(r_{m,t}-r_{t}^s)+\varepsilon_{i,t} \, \label{eq:CAPM} \end{align}

Despite of its attractive theoretical features, the estimation of $\alpha$ and $\beta$ using OLS is often problematic by the fact that the distribution of the error terms cannot be assumed to be independently identically normal and has a zero breakdown point, meaning that only one contaminated point can cause the estimator to take arbitrary values.

Consequently, different robust estimation approaches have been suggested like Least Absolute Deviations, M-estimators, GM-estimators, Least Trimmed of squares and Shrinkage robust estimators. We concentrated on Least Median of Squares.

The LMS estimator is defined as the solution to the following optimization problem:

\begin{align} \min_{\alpha,\beta}(\mbox{med}(\varepsilon_{i,t}^2)) \, , \label{eq:LMS} \end{align}

where $\varepsilon_{i,t} = y_{i,t} - \alpha - \beta x_{i,t}$ are the residuals of a factor model.

2 Estimation Results for CAPM

2.1 Implementation Details

For our application (LMS to CAPM) we consider data from the sample of publicly traded firms comprising the Dow Jones Industrial Average
for the period between 1970 and 2006.

We select two companies, IBM and ExxonMobil, based on the assumption that LMS estimation would perform better, resulting in lower conditional forecast errors when considering volatile stocks.

IBM estimation of beta, OLS and LMS

Click on the image to show the graph for estimates of $\beta$ for IBM in the period between 1971 and 2006.

2.2 Estimation Procedure

We use a rolling window of 200 days length moving from 1970 to 2006 day by day. For each given sample, the parameters $\alpha$ and $\beta$ were obtained by LMS estimation. In order to obtain high quality estimates, the heuristics have been restarted ten times for each given sample.2 We kept the estimates corresponding to the best value of the objective function.

Next, from this parameter estimates, we calculated the forecast of the excess return conditional on the market return for the next trading day.
The same calculation is done based on the OLS estimates. Finally,both forecast errors were compared.

2.3 Estimation Results

Despite the expectations that LMS estimators will be smoother, since they are not influenced by extreme observation, by definition, the
opposite result seems to occur, not only for IBM and Exxon but also for other stocks.

Given the differences in the estimates of $\beta$ obtained by LMS and OLS, we continue testing their relative forecasting
performance. For that we compare the mean squared forecast error (MSE) and the mean absolute forecast error (MAE), not only for IBM and
Exxon, but also for General Electric, Microsoft, General Motors and Boeing. In the case of the Exxon stock the MSE can be reduced substantially when using LMS instead of OLS and in the case of the IBM stock the OLS based forecasts result in slightly smaller errors both according to MSE and MAE. Still if we consider the MSE and MAE for the other stocks, the results can not be generalized.

2.4 Rate of Convergence

Besides the performance a LMS we are interested in comparing the performance of the two heuristic techniques. This is done by comparing their rate of convergence.

TA compared to DE, Best Result

$n_{p}$ $n_{G}$ $BestValue - DE$ $Var$ $Freq$
$50$ $100$ $4.9935-005$ $3.7887-028$ $30$
$n_{R}$ $n_{S_{\tau}}$ $BestValue - TA$ $Var$ $Freq$
$20$ $1000$ $4.9936-005$ $1.9420-015$ $1$
$200$ $200$ $4.9939-005$ $8.8922-016$ $1$

The results we obtain indicate the superiority of DE in terms of consistency and efficiency due to the fact that the search is run on a continuous search space.

3 Estimation Results for Fama and French Factor Model

The simplicity of the CAPM is also the reason that makes the model inadequate, as it uses a single risk factor to explain the returns on individual securities. In this section we consider a three-factor model, the Fama-French factor model, to capture the unexplained variation in excess stock returns.

The final form of the model is given by the following equation. While the market risk $r_{m,t}-r_{t}^s$ is the difference between market rate of return and risk free rate, SMB is the average returns on the three small portfolios minus the average return on the three big portfolios and HML is the average return on two value portfolios minus the average return on the two growth portfolios.

\begin{align} r_{i,t}-r_{t}^s=\alpha+\beta_{1}(r_{m,t}-r_{t}^s)+\beta_{2}SMB_{1,t}+\beta_{3}HML_{2,t}+\varepsilon_{i,t} \ \label{eq:APT} \end{align}

The estimation as well as the forecast procedure are consistent with those used for CAPM. The results of the application of the LMS
to the Fama/French multi-factor model are not fruitful and do not support the notion of superiority of LMS estimation.

4 Conclusions and Further Work

In this article we consider LMS estimator, as a robust method, for estimating the parameters of CAPM.Since LMS results in a search space with many local minima, standard methods cannot be used for its estimation. We consider heuristic optimization techniques which might be faster and, in particular, offer more flexibility when it comes to considering several explanatory variables or parameter constraints.

Particularly, the performance of two local search heuristic methods, DE and TA, is analyzed. It is clear that the estimation of the parameters of the CAPM using LMS is done more efficiently by DE due to the modeling of a continuous search space.

Its implementation allows a fast and reliable estimation of the CAPM by LMS. This is demonstrated by a rolling window analysis on a sample of two publicly traded firms, IBM and ExxonMobil, with daily data for the period between 1970 and 2006. LMS estimates do not exhibit less variation as might have been expected from the outlier related argument. Furthermore, when comparing the relative performance of both estimators in a simple one-day-ahead conditional forecasts, LMS estimator outperformed OLS only in the case of the Exxon stock.

The research can be extented by apply the method to different data sets, for example, stock returns from other stock indices or stock markets. Furthermore, it would be of interest to identify in more detail the situations when the estimation and forecast based on LMS outperforms OLS and vice versa (different sample lengths, high and low volatility periods).

Internal Links

Differential Evolution
Threshold Accepting
Related Articles

External links

1. Gilli, M. and P. Winker (2008). Heuristic optimization methods in econometrics. In: Handbook on Computational Econometrics. Elsevier. Amsterdam. forthcoming.
2. Rousseeuw, P. J. (1987). Robust Regression and Outlier Detection. John Wiley and Sons, New York.
3. Price, K. V., R. M. Storn and J. A. Lampinen (2005). Differential evolution: A Practical Approach to Global Optimization. Springer, Germany.
Click on the link below, which will take you to the COMISEF main website where the paper can be downloaded. The paper code is WSP-006.

Unless otherwise stated, the content of this page is licensed under Creative Commons Attribution-ShareAlike 3.0 License