A Quantitative Comparison of the Financial Returns of Index ETFs and Matched Index Mutual Funds

Investments in exchange traded funds (ETFs) have gained significant popularity among the financial investors. ETFs track industry-based indexes. Index mutual funds also track industry-based indexes. The financial investors do not have a documented analytical method to compare the financial returns of index ETFs and matched index funds. We have performed quantitative comparisons of the financial returns of index ETFs and matched index mutual funds with the use of nonparametric Kruskal-Wallis hypotheses tests, assuming that the financial returns of index ETFs and matched index mutual funds are independent. We have tested the null hypotheses that the medians of the distributions of the Sharpe ratios and the risk-adjusted buy and hold returns of the S&P 500 index, large cap index, mid cap index, small cap growth index, REIT index, and total bond market index ETFs are as the same as the medians of the distributions of the Sharpe ratios and the risk-adjusted buy and hold returns of the matched S&P 500 index, large cap index, mid cap index, small cap growth index, REIT index, and total bond market index mutual funds. The time periods of comparison have been from the inception dates of ETFs till the end of 2013. The findings from this research work suggest that we do not reject the null hypotheses that the distributions of the Sharpe ratios and the risk-adjusted buy and hold returns of index ETFs and the distributions of the Sharpe ratios and the risk-adjusted buy and hold returns of matched index mutual funds have the same medians during the time periods that we have tested.


Introduce the Problem
In the last few years, investments in exchange traded funds (ETFs) have gained significant popularity among the financial investors (Agapova, 2011;Charupat & Miu, 2013;Rompotis, 2009a;Sharifzadeh & Hojat, 2012).Charupat & Miu (2013) documented that by the end of 2011, the combined assets under management of all ETFs traded in the different stock exchanges around the word were $1.52 trillion.From 2001 to the end of 2011, there had been a 1400 percent increase in the assets under management of all ETFs (Charupat & Miu, 2013).According to these authors, approximately 4000 ETFs are now traded in 50 stock exchanges around the world.ETFs usually track broad industry-based indexes.Similar to ETFs, index mutual funds also track industry-based indexes.Index mutual funds were first introduced to the financial investors in 1972 (Agapova, 2011).According to this author, index mutual funds had more than $1 trillion of assets under management by the end of 2008 (Agapova, 2011).Index mutual funds can be considered as substitutes for index ETFs (Agapova, 2011, Sharifzadeh & Hojat, 2012).However, tax benefits are significant for investing in ETFs, because there are infrequent turnovers of stocks that comprise the portfolios of ETFs.Thus, the financial investors incur less capital gains taxes by investing in ETFs (Dimkpah & Ngassam, 2013).ETFs can be traded at any time during the stock exchange operating hours.ETFs are not characterized by high management fees.All of these characteristics of ETFs have led to the continuous growth in ETF investments (Dimkpah & Ngassam, 2013).

Explore the Importance of the Problem
Although in the past few years, investors have favored investments in index ETFs over index mutual funds, the problem that they face is how to compare analytically the financial returns of index ETFs with those of matched index mutual funds.Prather, Chu, Mazumder, & Topuz (2009) defined this problem as the dilemma faced by investors who want to invest in S&P 500 index was whether index mutual funds or index ETFs are better investment vehicles.In developing a quantitative model to compare the returns of index ETFs and index mutual funds, these researchers took into account bid-ask spreads and commissions that the financial investors incur to invest in ETFs.They concluded that ishares core S&P 500 index ETF was an appropriate investment for most investors than an index mutual fund (Prather et al., 2009).
There exists only one other prior study in the literature in which the researchers developed a quantitative method to compare analytically the returns of index ETFs and matched index mutual funds (Sharifzade & Hojat, 2012), although there was a serious flaw in their analysis.These researchers compared the Sharpe ratios and the risk-adjusted buy and hold returns of index ETFs with those of matched index mutual funds.A matched index mutual fund tracks the same index as the corresponding index ETF.The Sharpe ratio measures the return of an investment portfolio by taking into account the average annual return and the volatility of the return of the investment portfolio.Sharifzadeh & Hojat (2012) had used Wilcoxon signed-rank nonparametric hypothesis test in comparing the Sharpe ratios and the risk-adjusted buy and hold returns.Thus, they assumed tacitly that the Sharpe ratios of index ETFs and matched index mutual funds are dependent.They also assumed that the risk-adjusted buy and hold returns of index ETFs and matched index mutual funds are dependent.In reality, the Sharpe ratios of index ETFs and matched index mutual funds are independent.Similarly, the risk-adjusted buy and hold returns of index ETFs matched index mutual funds are actually independent.This was the flaw in their analysis (Sharifzadeh & Hojat, 2012).
In 1993, the predecessors of the current day ETFs, the Standard and Poors Depository Receipts (SPDR) first began trading on American Stock Exchange (Charupat & Miu, 2013).The goal of SPDR was to track the performance of S&P 500 index.The huge success of SPDR led to a surge in demand in the ensuing years for low-cost investment products consisting of baskets of investments that tracked equity based indexes (Charupat & Miu, 2013).In the early years (1990's), ETFs operated similar to index mutual funds.ETFs tracked industry-wide equity indexes, sector indexes, and fixed income indexes.In the recent years (late 2000's), the managers of innovative ETFs started to actively manage the funds and invest in commodities, such as, gold and silver, or employ leverages to generate returns that are positive or negative multiple returns of indexes these ETFs track (Charupat & Miu, 2013).
There are certain advantages of investing in ETFs, which are: low expense ratios, intraday trading, tax efficiency and transparency in costs (Charupat & Miu, 2013).At the same time, there are certain disadvantages of investing in ETFs that are: investors have to pay commissions, and pay bid/ask spreads when trading ETFs (Charupat & Miu, 2013).However, there is no commission to trade index mutual funds at the closing Net Asset Values (NAVs) on the stock exchange.The recently-introduced actively managed ETFs have high management fees, while leveraged ETFs require frequent trading involving commissions (Charupat & Miu, 2013).

Describe the Relevant Scholarship
One of the unusual characteristics of ETFs is the creation/redemption process, under which select traders can purchase or sell large units (creation units) of ETFs directly from or to the fund issuers at the NAVs (Net Asset Values).The creation/redemption process allows establishments of arbitrage bounds for market prices relative to NAVs.If the market price of an ETF is less than its value, then the traders can buy units of ETFs and sell those units for the underlying basket of securities, and gain from the price difference.Researchers in the past conducted studies the pricing efficiency of ETFs, and the role of arbitrage on minimizing the differences between market price and NAVs of ETFs (Charupat & Miu, 2013).Ackert & Tian (2000) and Elton, Gruber, Comer, & Li (2002) examined the pricing of SPDR units.The latter group of researchers had found that arbitrageurs take advantage of the price discrepancies, and thus, the prices discrepancies disappear within one day.Curcio, Lipka, & Thornton (2004) compared prices deviations of SPDRs to those of NASDAQ 100 group of trading stocks (QQQ) and found the average deviations are small for both ETFs.Engle & Sarkar (2006) studied price deviations of 21 ETFs based on US indexes.These researchers had found that the price deviations were small and within transaction costs and bid-ask spreads.In one of the earliest published work that compared the performance of index ETFs with that of index mutual funds, Kostovetsky (2003) indicated that the key areas of difference between the two investment vehicles were in management fees, shareholder transaction fees, taxation efficiency, and the qualitative factors such as, transaction convenience, short selling and ability to margin.Kostovetsky (2003) concluded that index mutual funds were better suited for small investors, while ETFs were preferable for large investors.
In the past, many researchers compared the returns of index ETFs and index mutual funds with the returns of the underlying indexes that these funds track.The difference in the absolute return of index ETF and the return of the underlying index that the ETF tracks is known as tracking error.Elton et al. (2002) documented that SPDR underperforms the S&P 500 index by an average of 28 basis points per annum from 1993 to 1998.They attributed the underperformance to the expense ratio of SPDR and the delay in reinvestment of dividends) from the constituent stocks) that are held in a non-interest bearing account until reinvestment.Over the same period, SPDR also underperformed index mutual fund by 18 basis points as documented by Elton et al. (2002).Gastineau (2004) and Blume & Edelen (2002) showed that the pre-tax performances of SPDR were lower compared to low cost index mutual fund due to inflexible replication strategy of SPDR in order to minimize the tracking error.Johnson (2008) found that the market segmentation of foreign country ETFs resulted in the tracking errors of those ETFs from those of corresponding index mutual funds in foreign countries.Aber, Li & Can (2009) compared the tracking errors of i share TM ETFs to the tracking errors of four Vanguard index mutual funds tracking the same indexes.They determined that the tracking abilities of ETFs differed from those of the corresponding index mutual funds based on the matches of the different pairs of ETFs and index mutual funds.They also found that the tracking errors of Vanguard index mutual funds are less than those of the ishare TM ETFs.Rompotis (2009a) Svetina (2010) studied ETFs from their inception to the end of 2007.This author concluded that on the average ETFs underperformed their benchmark indices and were not immune to tracking errors (Svetina, 2010).Svetina (2010) further concluded that only 17% of all ETFs directly competed with index funds, and these ETFs provided statistically indistinguishable returns from those provided by matched index funds.Blitz & Huij (2012) studied the tracking errors of US and European-listed ETFs that tracked the conventional broad emerging market indices.These authors found that these ETFs displayed much higher level of tracking errors than those ETFs that tracked developed market indices.Sharifzadeh & Hojat (2012) studied the financial performances of index ETFs and matched index mutual funds.These authors did not study the tracking errors of index ETFs and matched index mutual funds.Instead, they compared statistically the Sharpe ratios and the risk-adjusted buy and hold returns of index ETFs and matched index mutual funds.They used nonparametric Wilcoxon signed rank hypothesis test that assumed that the Sharpe ratios and the risk-adjusted buy and hold returns of index ETFs and index mutual funds are dependent.Although this assumption would be deemed erroneous, these authors concluded that 50 percent of the selected ETFs outperformed the corresponding index mutual funds.However, they also concluded that this outperformance was not statistically significant, and did not support their proposition that ETFs outperformed index mutual funds.

State the Hypotheses
The objective of the current research work is to overcome the limitation of the prior published work (Sharifzadeh & Hojat, 2012).We have assumed that the Sharpe ratios of index ETFs and matched index mutual funds are independent.We have also assumed that the risk-adjusted buy and hold returns of index ETFs and matched index mutual funds are independent.In the present research work, the author has used a nonparametric hypothesis test-Kruskal-Wallis test to compare c independent samples (Kruskal & Wallis, 1952).The prime advantage of this nonparametric hypothesis test is that it is not necessary to assume that the distributions of the Sharpe ratios and the distributions of the risk-adjusted buy and hold returns of index ETFs and the matched index mutual funds are normal or iie (independent and identically distributed random variables).Assuming that the populations differ only in centrality (i.e., location), the Kruskal-Wallis test compares the medians of c independent samples.This test requires that the populations be of similar shapes, but does not require normal populations.The findings from the present research work will help the financial investors to compare analytically the Sharpe ratios and the risk-adjusted buy and hold returns of index ETFs and matched index mutual funds.After reviewing the results from this research, the investors can determine whether investments in index ETFs are warranted because of superior returns, or because of other advantages in investment in index ETFs.
The first set of hypotheses to be tested in this study is: H o (Null): The distributions of the Sharpe ratios of index ETFs and matched index mutual funds have the same medians.
H 1 (Alternative): The distributions of the Sharpe ratios of index ETFs and matched index funds do not have the same medians.
The second set of hypotheses to be tested in this study is: H o (Null): The distributions of the risk-adjusted buy and hold total returns of index ETFs and matched index mutual funds have the same medians.
H 1 (Alternative): The distributions of the risk-adjusted buy and hold total returns of index ETFs and matched index funds do not have the same medians.Sharpe (1966Sharpe ( , 1994Sharpe ( , and 2007) developed a measure for the risk-adjusted portfolio performance, which is now widely used to determine the performances of mutual funds and ETFs.The Sharpe ratio for a fund SR f is defined as: SR f = (Mean return of the fund-RFR)/Standard Deviation of the return of the fund.

Method
RFR=Risk Free Rate, which is the average annual return on US Government ten year treasury bonds.
The Sharpe ratio takes into account the mean return of a mutual fund as well as, the volatility in the return as measured by the standard deviation of the return.The standard deviation of the return is used as a proxy to indicate the risk in investing in the fund.The Sharpe ratio measures the mean return of the fund in terms of how many standard deviation it is above or below the risk free rate.In other words, the Sharpe ratio is used to determine how well the return of the fund compensates the investor for the per unit risk that the investor takes.A higher value of the Sharpe ratio for a fund indicates a better financial performance of the fund (Rompotis, 2013).
In the past, other researchers developed hypothesis tests of Sharpe ratios using parametric methods that assumed normal distributions of financial returns (Johnson & Korkie, 1981;Memmel, 2003).Christie (2005) developed a model for an asymptotic distribution of Sharpe ratios.Almost all of these hypotheses tests were parametric in nature.Parametric hypothesis tests usually assume normal distributions and iid (independent and identically distributed random variables) of financial returns (Sharifzadeh & Hojat, 2012).Harwell (1988) demonstrated that using non parametric hypothesis tests would reduce the chances of Type I error, especially when sample sizes were small.The present author will use a nonparametric hypothesis test in order to reduce the chances of Type I error.We will test two sets of hypotheses to compare the financial performances of index ETFs and matched mutual funds: one set for Sharpe ratios and another set for risk-adjusted buy and hold returns.In calculating Sharpe ratios, the arithmetic average of the returns for the index ETF or matched index mutual fund will be used.We will gauge the total return by calculating buy and hold returns, which will involve compounding of the annual return over the years of comparison.
The following nomenclature has been used in our analysis: R et : Calendar-year total return for ETF e during year t, such as, t=2001,…2013 (for S&P 500 index ETF).
We have obtained each index ETF's calendar year total returns from Morningstar Web site.Morningstar calculates ETF's total return by taking into account the change in the fund's market price during the period, where all income and capital-gains distributions are reinvested during the period, and then dividing by the starting market price.R it : Calendar-year total return for index mutual fund I during year t, such as, t=2001,…2013( for S&P 500 index mutual fund).
We have obtained each index mutual fund's calendar year total returns from Morningstar Web site.Morningstar calculates mutual fund's total return by taking into account the change in the fund's Net Asset Value during the period, where all income and capital-gains distributions are reinvested during the period, and then dividing by the starting Net Asset Value.RABHTR i : Risk-adjusted buy and hold total return for index mutual fund i.

RABHTR i = BHTR i /Standard deviation of index mutual fund i
For both sets of hypotheses we have used Kruskal-Wallis non parametric hypothesis test, thereby assuming that the Sharpe ratios of index ETFs and the matched index mutual funds are independent of each other.We have also assumed that the total risk-adjusted buy and hold returns of index ETFs and matched index mutual funds are independent of each other.We have used 5% level of significance (risk of type I error) in conducting these hypothesis tests.The test statistic used for Kruskal-Wallis test is designated by H, where: The distribution of the sample H statistic is very close to that of the chi-square distribution with k-1 degrees of freedom when every sample includes at least five observations.This situation is true on our analysis.The p-value of H is calculated using the chi-square distribution with k-1degrees of freedom.

Sample Selection
For our research, we have calculated the Sharpe ratios and risk-adjusted buy and hold returns based on financial performances of S& P 500 index ETFs and matched S&P 500 index mutual funds between 2001 and 2013.This is because the inception dates of S&P 500 index ETFs were in 2000.The Sharpe ratios and risk-adjusted buy and hold returns of large cap, mid cap, small cap, REIT index ETFs and matched index mutual funds are calculated based on the financial performances of these funds between 2005 and 2013.The inception dates of these index ETFs were in 2004.For the comparison study of total bond market index ETF and matched total bond market index mutual fund, the Sharpe ratios and risk-adjusted buy and hold returns are calculated based on the financial performances between 2008 and 2013.The inception date of total bond market index ETF was in 2007.Therefore, we have dealt with small sample sizes; thus, we have not assumed normality in the distributions of the Sharpe ratios and in the distributions of risk-adjusted buy and hold returns of the index ETFs and the matched index mutual funds.

Results
Table 1 shows the descriptive statistics data of the pairs of index ETFs and the matched index mutual funds.Table 2 shows the comparisons of the mean Sharpe ratios and the mean risk-adjusted buy and hold returns of the pairs of index ETFs and the matched index mutual funds.Table 3 shows the decisions about the null hypotheses that the median Sharpe ratios and the median risk-adjusted buy and hold returns of the pairs of index ETFs and the matched index mutual funds are the same.

Discussion
As shown in the Table 1, the mean returns of index ETFs and matched index mutual funds are similar.The standard deviations of the index ETFs and matched mutual funds are also similar.The mean returns and the standard deviations of small cap growth index ETF and the matched index mutual fund are the highest in values.
The mean return and the standard deviation of the total bond market index ETF and the matched index mutual fund are the lowest in values.From the data in Table 2, we observe that the Sharpe ratio and the risk-adjusted buy and hold return values are the highest for the total bond market index ETF and the matched index mutual fund.This is because the standard deviation (or volatility) of the total bond market index ETF and the matched index mutual funds are the lowest in values.From Table 2, we also observe that the Sharpe ratio and the risk-adjusted buy and hold return values are the lowest for the S&P 500 index ETF and the matched index mutual fund.The Sharpe ratio and the risk-adjusted buy and hold return values are similar for the S& P 500 index ETF and matched mutual fund.We also observe that the Sharpe ratio and the risk-adjusted buy and hold return values are slightly different for the total bond market index ETF and matched mutual fund, probably because the time period of comparison for these funds is from only 2008 through 2013.
Based on the results presented in Table 3, we do not reject the null hypotheses that the Sharpe ratios of index ETFs for and matched index mutual funds have the same medians for all of the different indexes.This is because the calculated p-values are greater than the 5% level of significance.We also do not reject the null hypotheses that the risk-adjusted buy and hold returns of index ETFs and matched mutual funds have the same medians for all of the different indexes.This is because the calculated p-values are greater than the 5% level of significance in.This will mean that with 95% level of confidence, we can state that the investors will obtain similar median returns by investing in index ETFs or in matched index mutual funds.The investors may show preferences for investments in index ETFs because of the other preferable characteristics of index ETFs, such as, low expense ratios, intraday trading, tax efficiency and transparency in costs (Charupat & Miu, 2013).
Our results are similar to those obtained by Shrifzadeh & Hojat, (2012), when we have compared the Sharpe ratios of small cap growth index ETFs and matched index mutual funds.Specifically, like us, these authors found that there was no difference in the financial returns of small cap growth index ETF and matched index mutual funds.However, our results are different from those of Sharifzadeh & Hojat (2012), when we have compared the Sharpe ratios of REIT index ETFs, large cap index ETFs, and mid cap index ETFs, and matched index mutual funds in those categories.We have found that there is no difference in the Sharpe ratios of index ETFs and matched mutual funds in these categories, whereas, previous authors found that there were differences in Sharpe ratios of index ETFs and index mutual funds (Sharifzadeh & Hojat, 2012).We can conclude that as both of the index ETF and the matched index mutual fund track the same indexes, there cannot be any significant difference in the median values of the Sharpe ratios of the index ETF and matched index mutual fund.The strength of the findings of this research work is that it shows clearly that there is no statistically significant difference in the median Sharpe ratios of the index ETFs and matched index mutual funds.The financial investors can utilize these results in making judicious decisions in investing in index ETFs or in matched index mutual funds.
When we have compared the risk-adjusted buy and hold returns of all index ETFs and matched index funds, we have observed that our results are different from those of Sharifzadeh & Hojat (2012).These authors found that there were differences in the risk-adjusted buy and hold returns of the index ETFs and index mutual funds.On the other hand, we have found that is no difference in the risk-adjusted buy and hold returns of the index ETFs and the index mutual funds.We can conclude that as both of the index ETF and the matched index mutual fund track the same indexes there cannot be any significant difference in the median values of the risk-adjusted buy and hold returns of the index ETF and matched index mutual fund.The strength of the findings of this research work is that it shows clearly that there is no statistically significant difference in the median risk-adjusted buy and hold returns of the index ETFs and matched index mutual funds.These findings will be of importance to the financial investors when they decide to invest in index ETFs or in matched index mutual funds.
In this article we have compared the financial returns of index ETFs and matched mutual funds assuming independence of their returns.We have tested two sets of hypotheses: in the first set we have compared the Sharpe ratios.In the second set we have compared the risk-adjusted buy and hold returns of index ETFs and matched index mutual funds.The results of these hypotheses tests have indicated that we do not reject the null hypotheses that the distributions of Sharpe Ratios and risk-adjusted buy and hold returns of index ETFs and matched index funds have same median values.This means that the investors can expect to earn the same median values of financial returns, if they invest in index ETFs or in matched index mutual funds.Some limitations of our study could be: (1) we have compared the financial performance of S&P 500 index ETFs and matched index funds only from 2001 to the end of 2013; results may differ when the returns will be analyzed for longer time periods; (2) we have compared the financial performances of index ETFs (large cap, mid cap, small cap and REIT) and matched index funds only from 2005 to the end of 2013; results may differ when returns will be analyzed for longer time periods; and, (3) we have compared the financial performance of total bond market index ETFs and matched index funds only from 2008 to the end of 2013; results may differ when returns will be analyzed for longer time periods.
did a study on 20 Vanguard ETFs from 2004 through 2006, and 12 Vanguard index mutual funds from 1976 through 2006.He found that Vanguard ETFs and Vanguard index mutual funds provided similar returns and that both had low tracking errors.Rompotis (2009b) also studied the performances and trading characteristics of 73 i share TM ETFs from 2005 through 2006.He concluded that the tracking errors of the I share TM ETFs were correlated with the expenses and the underlying risks.
BHTR e : Buy and hold total return of RTF e for the time period such as, 2000-2013 (for S&P 500 Index ETF) BHTR e = (1+R e, 2001 )(1+R e, 2002 )(1+R e, 2003 )….(1+R e, 2013 ) RABHTR e : Risk-adjusted buy and hold total return for ETF e. RABHTR e = BHTR e /Standard deviation of fund e BHTR i : Buy and hold total return of index mutual fund i for the time period such as, 2000-2013 (for S&P 500 Index ETF).BHTR i = (1+R i, 2001 )(1+R i, 2002 )(1+R i, 2003 )….(1+R i, 2013 ) with k-1 degrees of freedom (k is the number of populations) ∑R k = sum of the ranks of index ETFs or index mutual funds, n k = size of sample k, and n=n 1 +n 2 +… +n k

Table 1 .
Descriptive statistics of the pairs of index ETFs and matched index mutual funds

Table 2 .
Comparison of the mean Sharpe ratios and the mean risk-adjusted buy and hold returns of pairs of index ETFs and matched index mutual funds

Table 3 .
Comparison of median Sharpe ratios and risk-adjusted buy and hold returns of pairs of index ETFs and matched index mutual funds