Empirical Analysis of Cost Estimation Accuracy in Procurement Auctions

This paper explores the impact of better cost estimation accuracy for a firm that bids on projects. Specifically, we investigate the effect of cost estimation accuracy on firms’ ability to submit lower bids amounts and their likelihood of winning in a bidding environment. We analyze highway construction bidding data between year 2001 and 2009 from a state department of transportation. Our results suggest that firms with more accurate cost estimation are more likely to lower their bids amounts and thus are more likely to win more bids than firms that have less cost estimation accuracy. The findings help us better understand the process and effect of cost estimation accuracy in the bidding environment and provide practical implications.


Introduction
Cost estimates are used by companies to make many operational and strategic decisions (e.g., Heitger, 2007;Ittner et al., 2002;Kaplan & Norton, 2001).It is a reasonable conjecture that a company that can estimate its costs more accurately than its competitors should enjoy a competitive advantage in the marketplace.These advantages may include the ability to price their products lower than competitors and thus win more orders.More accurate cost estimation can be achieved through a variety of approaches.These approaches include, for instance, a better cost management system like activity-based costing, a more formal and deliberate cost estimation system, and/or the acquisition of additional supplementary information regarding the project (e.g., Anderson, 1995;De Silva et al., 2008;Gupta & King, 1997;Ittner et al., 2002).However, as there is little evidence in the literature that more accurate cost estimates will necessarily lead to improvements in decisions or overall profits, several researchers have encouraged future studies to examine this relationship (Cooper & Kaplan, 1991;Callahan & Gabriel, 1998;Gupta & King, 1997;Heitger, 2007;Swenson, 1995).
In order to increase our understanding of the relationship between better cost estimates and improvement in decisions, we investigate the effects of cost estimation in a reverse auction setting (Note 1) using a large sample of data between year 2001 and 2009 from a state Department of Transportation (DOT) for publicly funded highway construction projects.The data includes 6,558 projects that were open to any company wishing to bid on the project.For each project, a set of companies submit bids to the DOT who then, in a public forum, open the sealed bids and award the contract to the lowest bidder.There are many other examples of companies that routinely participate in a reverse auction.For example, professional firms (e.g., accounting firms) bid on engagements, contractors bid on public and private construction projects, oil and gas companies bid on tracts for drilling rights, and businesses bid on procurement contracts to supply governmental agencies or other business.To measure cost estimation accuracy, we use standard deviation of bid's percent difference from the median bid as a proxy for cost measurement error.Our results suggest that on average, firms with more accurate cost estimates are able to lower their bids amounts and thus tend to win more bids than firms that have less cost estimation accuracy.
This study has important implications and contributions.First, despite the importance of cost estimate accuracy for many operational decisions, there is little empirical evidence that more accurate cost estimation leads to improvements in managerial decisions and competitiveness (e.g., Cooper & Kaplan, 1991;Gupta & King, 1997;Heitger, 2007;Swenson, 1995).Using bidding data from the highway construction industry, this paper contributes to our understanding of the importance of cost estimation accuracy in the bidding process.In particular, this study shows the importance of cost estimation on the ability of a company to submit lower bids and increase their likelihood of winning a project.
Second, this study has important practical implications for any company that routinely competes in reverse auctions.In the highway construction industry, many companies may not have an obvious cost advantage because they generally draw from the same labor pool, use the same suppliers, and use similar equipment.Therefore, estimating costs more accurately will reduce uncertainties associated with estimation errors thus allowing the company to reduce its price.Lastly, this study sheds light on prior research by using different research methodologies.Prior research in this area are mostly theoretical or/and experimental work (e.g., Callahan & Gabriel, 1998;Gupta & King, 1997;Heitger, 2007), while our study utilizes a large sample of empirical data from practice and provide evidence about the effect of better cost estimation in the bidding process.
The remainder of the paper is organized as follows.We review prior literature and develop our hypotheses in Section II.Section III describes our sample and variables, and presents our results of the hypotheses.Concluding remarks are offered in Section IV.

Literature Review and Hypotheses
Many studies in accounting literature emphasizes the importance of cost estimation accuracy, suggesting that accurate cost estimates are important inputs into numerous strategic and other managerial operational decisions (Anderson, 1995;Kaplan & Cooper, 1998;Heitger, 2007;Ittner et al., 2002;Kaplan & Norton, 2001).Gupta and King (1997) investigate the ability of a decision maker to learn from feedback in an experimental setting and make effective decisions even with imperfect cost reports.They find that, when participants learned from the feedback and updated their cost estimates making reports more accurate, profits increased.However, instead of directly testing the linkage between cost estimation accuracy and firms' profits, prior studies generally assume that more accurate cost estimates would necessarily lead to increased profits (e.g., Babad & Balachandran, 1993).In addition, there is also a lack of research as to whether cost estimation accuracy leads to improvement in other managerial decisions as we do in this study.Callahan and Gabriel (1998) examine the impact of accurate product cost information on firm profits in an imperfect market setting and suggest that the value of more accurate cost information may be dependent on the firm's competitive market structure.In a manner similar to Datar and Gupta (1994) and Christensen and Demski (1995), Callahan and Gabriel (1998) define cost estimation error as the variance of the cost report where those with low variance are considered more accurate.Using theoretical analysis, Callahan and Gabriel (1998) show that firms that compete on the basis of cost leadership (which may be characterized as a Cournot competition) will realize increased profits from more accurate cost estimates.However, firms that compete on the basis of product differentiation (operationalized as Bertrand competition), may not benefit from better product cost estimates.
Despite the importance of cost estimation accuracy acknowledged by prior research, there is little empirical evidence that more accurate cost estimation leads to improvements in decisions or overall competitiveness.Further, there is limited consensus among researchers about this linkage (Callahan & Gabriel, 1998;Gupta & King, 1997;Heitger, 2007).Prior studies encourage future research on whether accuracy of cost system could improve decisions, cost management and control, strategy formulation, and strategic management in organization (Gupta & King, 1997).This paper contributes to this area in understanding the importance of cost estimation accuracy to company's competitiveness.Specifically, we extend prior research by using empirical bidding data from the highway construction industry to explore the importance of accurate cost estimation in a procurement auction on the likelihood of submitting lower bids and likelihood of winning more bids.These two abilities are important in the bidding environment for to be profitable, a company must be able to submit bids that are low enough to be competitive yet high enough to mitigate the impacts associated with the winner's curse.
In this study, we use the median of all companies' bids for a particular project as a benchmark to measure the accuracy of cost estimates, as the median represents a consensus of project value (we will discuss this in details in the next section).The bid amount submitted by a company will include a markup added to the project cost estimate to account for desired profit as well as any informational uncertainties, i.e., cost estimation errors.Surveys (Gordon et al., 1981;Govindarajan & Anthony, 1983) as well as anecdotal evidence suggest that cost information is widely used to set prices, because costs can be reasonable surrogates for marginal costs upon which prices should be set based on expected utility theory (Hilton et al., 1988).We argue that companies that have more accurate cost estimation procedures (i.e., distributions closer to the median bidding value), for whatever reason, will submit lower bids.This is possible since the uncertainty surrounding the cost estimate is reduced thus any markups applied to a cost estimate doesn't have to compensate for this uncertainty (Hilton et al., 1988).At the same time, there will be a lower probability for these companies to lose money on a project compared to their competitors.That is, they will be less likely to experience the winner's curse.Therefore, if companies can estimate the cost of a project more accurately, they will tend to lower their bids amounts in order to win the project.This leads to our first hypothesis: Hypothesis 1: A better cost estimation accuracy (i.e., a tighter bid distribution around consensus values like median) will lead to lower bids amounts on average.
Our second hypothesis looks at the effect of company size on their bids amounts.We argue that larger companies usually have more expertise and experience in the bidding process as well as cost estimation resources, such as a larger group of engineering staffs, and thus have an advantage in bidding.That is, a lot of bids are based on historical bidding activities as well as knowing their competition.Here the competition is broad based -there are many companies in the business (367 firms in our dataset).Therefore, larger companies may tend to have better processes for cost estimation, that is, their bids amounts are more likely to be close to the consensus values.Therefore, we predict that: Hypothesis 2: A larger company will tend to have better cost estimation accuracy than a smaller company (i.e., a tighter bid distribution around consensus values like the median).
In addition, we investigate characteristics of companies that win bids compared to those that do not.A company can increase the probability of winning more bids just by bidding consistently low.However, in this case, they will be more likely to experience the winner's curse thus being less competitive in the market.Alternatively, if a company can estimate its costs more accurately, as indicated by a tighter bid distribution around the median, they may be able to bid lower with less concern of winning unprofitable bids, because they have more accurate estimates of costs than their competitors.Thus companies with more accurate cost estimation should win a higher proportion of the projects on which they bid, compared to those companies with less accurate cost estimation.This leads to our third hypothesis: Hypothesis 3: A company with better cost estimates (i.e., a tighter bid distribution around consensus values like median) will win more bids than a company with less accurate cost estimates.

Empirical Data
We collected the bidding data for highway construction contracts (between year 2001 and year 2009) from a state Department of Transportation website.Only projects which had four or more competing bids were included in our sample.Our sample is comprised of 6,558 usable projects that had bids submitted by 367 different companies.Table 1 presents some descriptive statistics for our sample.Panel A of Table 1 provides the frequency of bids amounts for different ranges.There are 382 bid observations (5.82% of the entire sample) that are more than $10 million, 802 bid observations (12.22% of the sample) that are more than $5 million, 2791 bid observations (42.56% of the sample) that are more than $1 million, and 5356 bid observations (81.67% of the sample) that are more than $250,000.We also present the number of bids by firm in Panel B of Table 1.There are 19 companies that bid more than 100 projects, accounting for 5.18% of the sample.There are 116 companies that bid more than 10 projects, accounting for 31.61% of the sample.), the mates, er bid proxy perts, xpect e, we alues ubmit i.e., a el A: more y just submit a large number of higher bids in the hope that they win a contract with as much profit as possible.This lack of support may also be due to our use of proxy for the size of a firm which may not accurately reflect company size (Note 2).Hypothesis 3 (H3) predicts that better cost estimates (i.e., a tighter bid distribution around the median) will lead to winning more projects than those companies with less accurate cost estimates.We show that a tighter bid distribution around consensus values (median) will lead to lower bids on average (supported by H1).
Theoretically, if a company has a lower bid, it will have a higher probability to win the bid (since the lowest bid amount wins.)Our results suggest that H3 is marginally supported (t = -1.64,p-value = 0.056, one-tailed).As presented in Table 4 and Figure 4, when companies have a smaller standard deviation of percent difference from the median (i.e., a tighter distribution around consensus values which is indicative of better cost estimation), they tend to have a relatively higher probability of winning the bid.

Figure 3 .
Figure 3. Cost estimation accuracy* as a function of company size (test for H2) *: The cost estimation accuracy is measured as standard deviation of percent difference from the consensus value, median bid.

Table 1 .
Descriptive data

Table 3 .
Cost estimation accuracy* as a function of company size (test for H2)

Company size is measured by average bid amount by the company
*:The cost estimation accuracy is measured as standard deviation of percent difference from the consensus value, median bid.**: All p-values are two-tailed unless otherwise noted.

Table 4 .
Percent of winning bids of bids submitted by each company as a function of cost estimation accuracy* for each project (test for H3) *:The cost e