Random Walk or Switching Regimes in Stock Prices? Evidence from Out-of-Sample Forecasts

We use monthly observations on general stock price indices, over January 2001–August 2013, in order to assesssimple stochastic time series models in terms of out-of-sample forecasts. Specifically, we examine the relativestrength of out-of-sample forecasts of a random walk, with and without drift, against that of a non-linearsegmented trends model where the switch between states is governed by a Markov chain. The forecastingperformance of these processes is assessed by the root mean squared error of short- and long-term out-of-sampleforecasts, varying from 1- to 12-month horizons. We obtain compelling evidence in favor of the Markovswitching process in forecasting stock prices over short and medium-term horizons and across all countriesconsidered. These results are most likely due to risk averse behavior of investors which has been amplified bythe recent financial crisis.


Introduction
Since Samuelson's (1965) seminal work on the randomness of stock price changes, unforecastability of share prices has been associated with the random walk or martingale property of the relevant series.This means that forecasts based on other stochastic processes or structural models are not expected to beat random walk forecasts.Even though early empirical works largely verified this result, Granger (1992) reviewed a number of studies that employed variants of switching regimes models and reported evidence of improvement in out-of-sample forecasts over naïve random walks.In these approaches regime shifts are explicitly determined by changes in some variables or moments and are not inferred by the whole information set.In other words, these models do not rely on the data to infer when such regime shifts occur.Nevertheless, there is more recent work that appears to provide evidence in favor of naïve model forecasts over ARIMA representations of stock prices (e.g., Grigaliūnienė, 2013).Thus, the empirical record on stock price forecastability seems to be rather mixed and it is worth revisiting the issue using recent data that span the period before and after the financial crisis of 2008.
In this paper, we re-examine the relative forecasting strength of a random walk for stock prices, with or without drift, against that of a non-linear Markov switching process, using recent monthly data from 8 different countries and the Eurozone.In particular, the forecasting ability of these processes is compared in terms of the root mean squared error of out-of-sample forecasts obtained from monthly observations on the real stock price indices of the Eurozone, Germany, Greece, Ireland, Portugal, Spain, Japan, the UK and the US over the period from January 2001 until August 2013.We opt for out-of-sample forecasts since, as argued by Granger (1992, p. 11), a forecasting model has to show that it can actually forecast and it is not sufficient to assess the model by in-sample forecasts.As he puts it, 'only out-of-sample evaluation is relevant'.Also, except for the Eurozone and Germany, we examine European countries that have been under severe fiscal consolidation programs advanced by a three-party support scheme (European Commission, European Central Bank, International Monetary Fund), also known as troika, in the context of Memoranda of Understanding (MoU).In so doing, we seek to determine whether our results are influenced by the massive fiscal adjustment and the ensuing recessions that took place after the outbreak of the 2008 financial crisis in countries under MoUs like Greece, Portugal, and Ireland, and in countries that chose drastic fiscal consolidation outside troika's supervision like Spain.Such dramatic and unexpected changes in macroeconomic policies may be readily thought of as major regime shifts that are likely to confuse the naïve model but not necessarily a model that allows for structural breaks determined by the data history.Indeed, our empirical results suggest that, for short-term horizons up to six months, the superiority of the switching regimes out-of-sample forecasts is overwhelming regardless of country or forecast window.Even for longer horizons (9 or 12 months) the relative strength of the Markov forecasts holds up in most cases.Thus, we obtain strong evidence against the random walk property of stock prices but this does not mean that markets are not efficient as these results may be due to risk averse behavior which is not accounted for by the processes considered here.
In the second section we discuss the models and forecasting techniques and in the third section we report the empirical findings.A discussion and concluding remarks are presented in the final section.

Methodology
If p t (t = 1, 2, …, T) denotes the real price of stock at time t and d t the first difference of p t (d t = p t -p t-1 ), then p t is said to follow a random walk if it has the martingale property: where I t denotes the information set that includes all past values of p up to time t.By the law of iterated expectations, the out-of-sample forecast of the stock price k-periods from now is: If p t is represented by a random walk with drift and d t denotes the first difference of p t (d t = p t -p t-1 ), the time-t forecast of p t+k is: where , and n (n<T) is the size of a sub-sample which is used for the estimation of the model.
An alternative model allows the mean of the change in prices to take different values across different states: where s t is an unobserved state variable that takes on discrete values {1, 2}, and u is an error term.State 1 can be considered as an expansion state while state 2 can be thought of as a recession state.Apparently, equation ( 4) allows for switching means and variances and the law of motion of the state variable s t will be assumed to be a Markov chain with stationary transition probability matrix: where p ij =Pr(s t =j|s t-1 =i), i, j = 1, 2. Hence, the time-t forecast of d t+k is (see e.g.Hamilton, 1993;Kirikos, 2000Kirikos, , 2013)): where D t includes all the values of d up to time t,  t  = [Pr(s t =1|D t ) Pr(s t =2|D t )] is the vector of filter inferences at date t (see Hamilton, 1990Hamilton, , 1993)), that is, the vector of the probabilities that the process is in a particular state based on information up to that time, and  s  = [ 1  2 ] is the vector of means across states.The probabilities  t  are nonlinear functions of the data and, thus, forecasts given by equation ( 6) are nonlinear.Also, here lies the main difference of the current Markov model relative to other switching regimes approaches.That is, by estimating the probabilities  t  we use the data in order to infer a switch in regime instead of imposing such a shift in terms of some arbitrary measure.

Estimates of the parameters (
, p 11 , p 22 ) are taken by maximizing the sample likelihood function via the EM algorithm (see Hamilton, 1990).
Using (6), we take forecasts of the real stock price index by: In all cases, out-of-sample forecasts are based on recursive estimation of the models (see Rogoff & Stavrakeva, 2008).In particular, we start with a sub-sample of size n, estimate the model and compute the first k-period-ahead forecast.Then, we expand the sub-sample by including the next available observation, so that the new sub-sample has n+1 observations, estimate the model again and compute the next k-period-ahead forecast.This iteration goes on until the size of the sub-sample used for out-of-sample forecasting becomes T-k, where T is the full sample of observations.
The square root of the mean squared error (RMSE) of out-of-sample forecasts is: Probably, the smaller the RMSE the better the forecasts.However, it should be noted that the results reported below do not change when forecasts are compared in terms of the mean absolute error (MAE).

Empirical Results
The empirical results are based on monthly observations on the real stock prices of the Eurozone, Germany, Greece, Ireland, Portugal, Spain, Japan, the UK, and the USA, for the period January 2001-August 2013 (152 observations).Computations are based on GAUSS code and the appendix contains a description of the data and their sources.
The graphs below depict the RMSE of forecasts for 1-, 3 Apparently, Markov out-of-sample 1-to 6-month forecasts by far outperform naïve random walk forecasts in terms of the RMSE criterion, for all countries and forecast windows considered.What is more, naïve short-term forecasts provide rather poor predictions of future stock prices as opposed to Markov forecasts which exhibit little average deviation from actual prices.Even for longer horizons (9 or 12 months) the Markov segmented trends model provides better out-of-sample forecasts for most countries, with the exception of the post-sample periods after 2008 and 2010 for some countries (Euroarea, Germany, Greece, Portugal and the UK).In any case, the relative strength of switching regimes forecasts declines for long-term predictions.
A closer look at the results suggests that as the post-sample period is shortened, long-term (9-or 12-month) random walk forecasts improve faster than similar Markov forecasts, thus reducing the relative strength of the latter or, in some cases, outweighing their superiority.When this finding is combined with the fact that our forecasts are the outcome of recursive estimation of the models, it seems likely that the starting point of the financial turmoil in September 2008 presents a major shift in the stock price series that confuses the naïve random walk model.However, as we move away from this point the series evolve more smoothly and the random walk model regains part of its forecasting competence.Also, the similarity of results across countries suggests that regime shifts associated with the Great Recession have uniformly influenced stock prices regardless of any particular fiscal policies such as those pursued in European countries under MoUs.

Concluding Remarks
Using monthly data on real stock prices of 8 different countries and the Eurozone, over the period 2001-2013, we obtained overwhelming evidence that a Markov switching regimes model consistently outperforms a random walk, with or without a drift parameter, in terms of short-run 1-to 6-month out-of-sample forecasts.These results are invariable with respect to the forecast window used and with respect to drastic differences in fiscal policies pursued by different countries.Also, longer-term 9-to 12-month out-of-sample Markov forecasts beat naïve forecasts in most cases, and the faster improvement of the random walk predictions, as the forecast window is narrowed, is most likely due to the major shift of the real stock price series that took place with the outbreak of the financial crisis in September 2008.
The failure of the random walk model to represent real stock prices may be interpreted as a possibility of making endless profits in stock markets since prices seem to be forecastable.However, this isn't so because markets may be efficient even when prices are forecastable.What is required for market efficiency is that risk-adjusted prices and returns are unforecastable.Indeed, in dynamic stochastic general equilibrium models, the random walk property of stock prices results under risk neutrality (see Sargent, 1987, p. 94) and such behavior is not expected to characterize investors over the recent financial turmoil.On the contrary, under bubble collapses in financial markets, risk averse behavior is most likely heightened.In that vein, Kirikos (2009) provided evidence that, under a linear error correction model for the information variables, investors in the Greek stock market have exhibited rather strong risk aversion, even before the financial crisis, and that statistical rejections of the efficient market model are rather due to transitory deviations.Thus, our current results suggest that an empirical re-assessment of the efficient market model for stock prices under risk aversion and a non-linear Markov switching regimes representation of stock prices and dividends is a promising avenue for future research.
-, 6-, 9-and 12-month horizons and for three different forecast windows; September 2006-August 2013, September 2008-August 2013, and September 2010-August 2013.Our forecast windows begin in September because of the outbreak of the recent financial crisis in September 2008 which may have marked a major shift in the series.RW(dr) denotes the random walk with a drift, RW denotes the random walk without drift, and 'Markov' stands for the Markov switching regimes process.