Stock Market Prediction Performance of Neural Networks : A Literature Review

In this paper, previous studies featuring an artificial neural networks based prediction model have been reviewed. The main purpose of this review is to examine studies which use directional prediction accuracy (also known as hit ratio) or profitability of the model as a benchmark since other forecast error measures namely mean absolute deviation (MAD), root mean squared error (RMSE), mean absolute error (MAE) and mean squared error (MSE) have been criticized for the argument that they are not able to actually show how useful the prediction model is, in terms of financial gains (i.e. for practical usage). In order to meet the publication selection criteria mentioned above, a large number of publications have been examined and 25 of papers satisfying the criteria are selected for comparison. Classification of the eligible papers are summarized in a table format for future studies.


Introduction
According to the Efficient Market Hypothesis (EMH), stock prices cannot be forecasted by investors since markets reflect all of the currently available information.From this point of view, it is suggested that stock prices proceed in a stochastic manner.This idea is also known as Random Walk Hypothesis (RWH).Conversely; it has been suggested for a long time that prices can be predicted using different kind of techniques mainly classified as time series forecasting models.As a matter of fact, there is no certain consensus on which hypothesis is actually more likely to be relied on.However, a large number of studies empirically proved that prices can be predictedat least to a certain degree -using different methods.For example, (Brock, Lakonishok, & LeBaron, 1992) investigated predictability of the Dow Jones Industrial Average index by using two technical trading rules namely moving averages and trading-range breaks.Using these two trading rules, they generated buy and sell signals.Their results provide strong support for the technical strategies.Especially recent studies which employ artificial (computational) intelligence methods such as artificial neural networks (ANN), support vector machines (SVM), genetic algorithms (GA) etc. suggest that significant levels of market inefficiency is present in a wide range of markets hence predictability of prices is viable.
Forecasting in the financial time series is basically predicting the behavior of one step ahead of the series with the help of various variables.Similarly, it would not be wrong to make the same generalization for stock price estimates.In finance practice, stock price prediction/forecasting efforts generally fall one of the two categories in terms of explanatory variables namely fundamental analysis and technical analysis.Techniques from both categories are also used by forecasters simultaneously for improving forecasting ability.Furthermore, there have been numerous time series forecasting models of statistical nature which employ variables from fundamental and technical analysis suggested by scholars.There are also a growing number of papers in the literature employing an artificial intelligence technique purely or combined with other statistical techniques.One of the most predominantly preferred and also in widespread use in the industry is ANN.
When employing ANN in prediction, selection of input variables for forecasting is as crucial as the topology of the ANN.It has been shown in many studies that the same model can produce significantly different outcomes when fed with different inputs i.e. independent variables.Thus another main purpose of this review is to examine studies which use directional prediction accuracy or profitability of model as a benchmark since from the practical point of view it is the main objective of the prediction of financial time series.A prediction with little forecast error (measured as MAD, RMSE, MAE, and MSE) does not necessarily translate into a capital gain (Leung, Daouk, & Chen, 2000).The practical aim of forecasting is the profits generated from a successful sequence of trades or financial gains based on prediction results.It does not matter whether the forecasts are accurate or not in terms of normalized mean squared error (NMSE) or gradient (Yao & Tan, 2000).For example (O"Connor &Madden, 2006) and(De Faria, Albuquerque, Gonzalez, Cavalcante, &Albuquerque, 2009) found that there is a disparity between RMSE and profitability of the ANN model.Which means that obtaining low RMSE does not provide high returns, in other words the relationship is not linear between two.Moreover, correct directional predictions and profit-based performance metrics is also easy and practical to draw interpretations on the capability of the underlying prediction model.Hence, in this paper it is intended to classify studies not only for their model selection criteria but also for the inputs used for the prediction and also how accurate is using them in terms of predicting directions.In this survey, we will consider studies which use percentage of profit-generating or in other terms percentage of winning trades benchmark measures for testing the suggested model.From this point of view, this survey"s genuine approach is compare previous models in literature for their explanatory/input variables used for prediction and how accurate they are in predicting the direction of the related time series.Therefore the aim of this study is to put forward the importance of input selection as well as the model selection and give insight to researchers and practitioners.
There are other review studies on artificial intelligence and ANN based financial forecasting methods such as (Bahrammirzae, 2010), (Rather, Sastry, & Agarwal, 2017), (Zhang, Patuwo, & Hu, 1998), (Adya & Collopy, 1998), (Paliwal & Kumar, 2009), (Atsalakisa & Valavanisb, 2009).For example, (Bahrammirzaee, 2010) reviewed comparative studies where ANN, expert systems (ES) and hybrid systems were compared each other and also with traditional statistical methods.(Rather et al., 2017) described a more general framework by separating studies based on single asset prediction models (which contains autoregressive moving average, singular and hybrid models) with portfolio selection models.(Paliwal & Kumar, 2009) reviewed comparative studies of multilayered feedforward neural networks and statistical techniques used for prediction and classification in the areas of accounting and finance, health and medicine, engineering and manufacturing, marketing, general applications.(Zhang et al., 1998) summarized modeling issues of ANN forecasting and reviewed studies comparing ANN with traditional statistical methods based on predicted variables.

Classification of Articles
In this review, a large number of publications were examined but only a small number of them considered to meet the criteria expressed before.For each publication, four categories are specified.Those categories are model, forecasted index and predicted time interval, input variables, and result categories.In the "model" category, prediction model(s) proposed by authors and other models for comparison are listed.The other category namely "forecasted index and predicted time interval" is considered since market conditions like developed markets, emerging markets and, frontier markets are important parameters of prediction and also the length of estimation (also known as test period) is a required feature for testing robustness of the model.As mentioned before, input or exploratory variables are quite important parameters for a prediction model because the predictive power of the model is largely dependent on the inputs used hence the third category.The last category which is essential to our survey for comparing studies in terms of correct directional prediction or return (profit) obtained by using proposed prediction models is the "result" category.All of the reviewed papers are summarized in Table-1 based on their qualifications at each category.(Niaki & Hoseinzade, 2013) used 27 financial and economic factors as inputs for feed-forward neural networks in order to forecast direction of Standard & Poor's 500 (S&P 500).They followed a buy-and-sell strategy which is determined by the direction of the market.Due to their proposed strategy, portfolio is rearranged according to the ANN"s forecast.They found that ANN performs better than passive buy-and-hold strategy and also outperforms the logit model.(Kara, Boyacioglu, & Baykan, 2011) developed an ANN and SVM using ten technical indicators as inputs and then compared their performances in predicting the direction of movement of the daily Istanbul Stock Exchange (ISE) National 100 Index.Their output of the ANN network was two patterns (0 or 1) of stock price direction.They showed than ANN shows better performance than SVM.(Yao, Tan, & Poh, 1999) using some technical indicators as inputs, applied several back-propagation neural networks (BNN) in order to predict the KLSE stock market index and compared the returns earned by BNN with conventional ARIMA models.Their results show that the neural network model can get better returns compared to conventional ARIMA models.(Jasic & Wood, 2004) derived buy and sell signals from single hidden layer neural network predictions which uses lagged values of S&P 500, DAX, TOPIX and FTSE index as inputs and found significantly different from unconditional one-day mean return which can provide significant net profits for plausible decision rules and transaction cost assumptions.(Fernandez-Rodriguez, Gonzalez-Martel, & Sosvilla-Rivero, 2000) compared the profitability of back-propagation learning rule based artificial neural networks with a simple buy-and-hold strategy in General Index of the Madrid Stock Market.Their model receives 9 previous days" returns as input and scales output between [-1, 1] interval.As a result it is asserted that except for ""bull"" markets, in absence of trading costs, the technical trading rule is always superior to a buy-and-hold strategy.(O"Connor & Madden, 2006) compared different ANNs with different settings in predicting movements in the Dow Jones Industrial Average index.They conducted six experiments using feed-forward ANN.In each experiment different input setups are tested.Accordingly, in some of the experiments external factors (such as currency data and crude oil) haven"t been taken into account as inputs, instead Dow Jones time series data and related technical indicators have been taken as inputs.The results have shown that using external indicators as inputs, the overall performance in terms of profitability and directional success of the model has improved significantly.(Chen, Leung & Daouk, 2003) favored the idea that forecasting the direction of price changes rather than price levels and used probabilistic neural networks in order to forecast the direction of index returns.Using the obtained forecasts of the direction of returns they employed two trading strategies called "single threshold triggering" and "multiple threshold triggering".Then the authors compared the results with simple buy and hold strategy, random walk models and GMM-Kalman filter models.(De Faria et al., 2009) predicted the directions of the principal index of the Brazilian stock market with ANN and adaptive exponential smoothing (AES) method where different settings tested for both ANN and AES and concluded that the AES method did not contribute to predict the correct sign of the return.On the other hand ANN and AES produced almost the same RMSE.(De Oliveira, Nobre, & Zárate, 2013) conducted a domain analysis to be informed about financial market and to identify variables that drive stock prices.Employing resilient back-propagation algorithm for network training, they forecasted Petrobras stock PETR4 time series with ANN.(Huang, Nakamori, & Wang, 2005) conducted a comparative study where predicted weekly movement direction of NIKKEI 225 index results obtained by SVM, Elman backpropagation neural networks (EBNN), random walk model (RW), linear discriminant analysis (LDA), quadratic discriminant analysis (QDA) and a combining model of SVM with other classification methods compared each other.(Kumar & Thenmozhi, 2006) is another study of forecasting the direction of S&P CNX NIFTY Market Index with various methods.LDA, logit model (LM), ANN, Random Forest (RF) and SVM are compared each other.(Leung et al., 2000) compared linear discriminant analysis, logit, probit, probabilistic neural network, exponential smoothing, multivariate transfer function, vector auto regression with Kalman filter, and multilayered feedforward neural network in predicting daily direction of S&P 500, FTSE 100, and Nikkei 225.(Zhong & Enke, 2017) employed principal component analysis (PCA), fuzzy robust principal component analysis (FRPCA), and kernel-based principal component analysis (KPCA) for dimension reduction of 60 financial and economic variables.Following this, ANNs are used with the pre-processed data sets to forecast the daily direction of S&P 500 Index ETF.(Asadi, Hadavandi, Mehmanpazir, & Nakhostin, 2012) proposed a hybrid intelligent model which is combined of genetic algorithms and Levenberg-Marquardt (LM) algorithm with ANN and tested on Taiwan Stock Exchange index (TSE), Tehran Stock Exchange Prices Index (TEPIX), Index of top 50 Companies, Industry index, Index of Financial Group, Dow Jones Industrial Average Index Series, and Nasdaq Index Series.(Lee & Lim, 2011) utilized a neuro-fuzzy system which is a supervised classification technique named neural network with weighted fuzzy membership function (NEWFM) and applied on Korea composite stock price index (KOSPI) data.(Dai, Wu, & Lu, 2012) (Kim, 2003) applied support vector machines (SVMs) to forecast the daily Korea composite stock price index (KOSPI) and compared it with back-propagation neural networks and case-based reasoning.(Mingyue, Cheng, & Yu, 2016) optimized the ANN model using genetic algorithms (GA) to forecast the Japanese stock market index and compared results with other studies.(Kim & Han, 2000) employed genetic algorithms (GAs) to assign values of weights by simultaneous optimization of connection weights for artificial neural networks (ANNs) and to feature discretization, then they forecasted the daily Korea stock price index (KOSPI) with proposed hybrid model.They compared three models with each other.These are linear transformation with the back propagation neural network (BPLT), linear transformation with ANN trained by GA (GALT) and, GA approach to feature discretization (GAFD) for ANN.

Review of Literature
In the comparison table best results obtained by authors are listed.Also, in the results column, if one study has both, percentage of correct directional predictions and returns obtained at some transaction costs performance measures, former is preferred.Input variables: 4 (for each time series) Economic Variables (4): First difference of 3-month T-bill rate for the US, and first difference of call money rate for the UK and Japan; First difference of long term government bond rate for the US, first difference of 20-year government bond rate for the UK, and first difference of long term government bond rate for Japan; First difference of consumer price index for the three countries respectively; First difference of industrial production for the three countries, respectively Only 4 of the 25 papers listed in the above table have been identified as favoring the return rate of the underlying model as the performance measure, while the remaining 21 have been identified as papers which measure the performance of the proposed model as the percentage of correct directional predictions.Another reason for using the selection criteria mentioned before, is the fact that surveyed papers in this study have been using the same performance measures.Thus this gives a naturally appropriate bed for comparing them with each other.

Conclusion
ANN is known to be employed in a wide range of application areas among which different business disciplines come first.Financial prediction is one such field in which ANN is used alone or in combination with different machine learning techniques.In this survey, selected papers which exploit ANN for making financial time series prediction have been reviewed based on certain criteria.These criteria are basically the usage of statistics concerning return rate of the investment made in a financial market or percentage of correct directional predictions of the underlying ANN based prediction model.To sum up, reviewed papers mostly suggest that ANN combined with another statistical or machine learning technique yield better results.Moreover, a preliminary analysis using multivariate statistical techniques on data sets that would be fed to ANN promise a more profitable set of hybrid models.Thus, promoting hybrid models wouldn"t be unwise in case of financial time series predictions.
combined nonlinear independent component analysis (NLICA) and neural networks to forecast some Asian stock markets.Using NLICA they transformed raw data into independent components which are served as input variables of the neural network.(Lu& Wu, 2011)  proposed cerebellar model articulation controller neural network (CMAC NN) and compared it with support vector regression (SVR) and a back-propagation neural network (BPNN) in forecasting Nikkei 225 and Taiwan Stock Exchange (TAEIX).(Yu, Wang, & Lai, 2009) improved a neural-network-based nonlinear metamodeling technique to forecast S&P 500, NYSE, and US dollars vs. Euros (EUR) and US dollars vs. Japanese yen (JPY) exchange rates.(Chao, Li-li, & Ting-ting, 2012) developed a new support vector machine (SVM) based on wavelet kernel function which is a combination of SVMs and wavelet kernel function.Prediction results on NASDAQ composite index of Polynomial kernel SVMs, Gaussian kernel SVMs, Morlet wavelet kernel SVM, Gaussian wavelet kernel SVM, and Biorthogonal spline wavelet Bior (4.4) kernel SVM are then compared each other.(Lu,Lee, & Chiu, 2009)  first used independent component analysis (ICA) to generate the noiseless independent components and then served them as inputs to the support vector regression for financial time series forecasting.(Wang, Wang,Zhang, & Guo, 2012)  combined the exponential smoothing model (ESM), autoregressive integrated moving average (ARIMA), and the back propagation neural network (BPNN) to forecast the closing of the Shenzhen Integrated Index (SZII) and opening of the Dow Jones Industrial Average Index (DJIAI).(Kao, Chiu, Lu, & Yang, 2013) used nonlinear independent component analysis (NLICA) to extract features (independent components) from forecasting variables then used them as inputs of support vector regression (SVR) to forecast Shanghai Stock Exchange Composite (SSEC) and Nikkei 225 stock indexes.

Table 1 .
Results of reviewed articles Current day"s Dow Jones opening value, Previous 5 days" Dow Jones opening values, Previous 5 days" Daily Dow Jones Gradients, Previous 5 days" WTI Cushing crude oil price Previous 5 days of the USD/YEN exchange rate, Previous 5 days of the USD/GBP exchange rate, Previous 5 days of the USD/CAN exchange rate ): Exchange rate between USD-British pound, USD-Canadian dollar, USD-Japanese yen, Exxon Mobil stock return in day t-1, General Electric stock return in day t-1, Microsoft stock return in day t-1, Procter & Gamble stock return in day t-1, Johnson and Johnson stock return in day t-1.
Due to lack of space, reader is referred to the original paper to see each of the raw financial and economic variables.
Three previous day"s futures closing prices of Nikkei 255 traded on SGX-DT, OSE and CME, and the previous day"s cash market closing index Input Variables (SSEC index closing price) 4: Basic Price Data (2): The previous day"s cash market closing prices, and the current day"s opening cash index Technical variables (2): The previous day"s cash market high and low Technical variables (12): %K., %D, Slow %D, Momentum, Price rate-of-change, Williams" %R, A/D Oscillator, Distance of current price and the moving average of 5 days, Distance of current price and the moving average of 10 days, Price oscillator (OSCP), Commodity channel index, Relative strength index Technical variables (13): Stochastic %K, Stochastic %D, Stochastic slow %D, Momentum, ROC, LW%R, A/O Oscillator, Disparity in 5 days, Disparity in 10 days, OSCP, CCI, RSI Input Variables (Type II inputs) 8: Technical variables (8): On Balance Volume (OBV), Bias Ratio (BIAS 6 ), Ratio of the number of rising periods over the 12 day period (PSY 12 ), Average return in the last n days (ASY 5 , ASY 4 , ASY 3 , ASY 2 , ASY 1 )