Examining the Yahoo ! Sponsored Search Auctions : A Regression Discontinuity Design Approach

The sponsored search auction is a successful pricing mechanism which helps search engine companies sell navigation service to advertisers and generate multibillion dollar revenue. For the two popular sponsored search auctions—the Generalized First Price (GFP) auction and the Generalized Second Price (GSP) auction—current consensus in both the industry and academia is that the GSP auction is more stable than the GFP auction. Specifically, in the GSP auction, bidders are less likely to “game the system”, meaning that an individual bidder will change his bid less frequently and his bid range will be smaller. This paper examines this prevailing belief using a Regression Discontinuity Design (RDD) approach and finds that after bidders switch to the GSP auction, they actually change their bid 19% more frequently and increase their daily bid range by 46%. The paper suggests that the prevailing automated bidding strategies should not be ignored in explaining bidders’ behavior.


Introduction
The sponsored search auction has played an indispensable role in the success of search engine giants like Yahoo! and Google.For example, Yahoo!'s first half-year revenue in 2008 was $3.62 billion, and at least 50% of that revenue came from the sponsored search auction.For Google, its first half-year revenue in 2008 was $10.55 billion, with 97% of this revenue generated by the sponsored search auctions.Actually, the sponsored search auction is not only crucial to search engine companies, but it is also "vital to the success of many other small businesses" such as bid management software firms, bidding campaign consulting firms, and key word selecting firms, etc. (See Jansen and Mullen, 2008).
The sponsored search auction is a pricing mechanism which helps search engine companies sell navigation services to advertisers.When addressing search requests, search engines display both the search results and advertisers' web links, which are called sponsored links.These sponsored links attempt to navigate potential customers to specific product web sites.Because this targeting of potential costumers has proven effective, advertisers are willing to pay in order to obtain an ideal placement for their web link on a search result page.Search engine companies invented the sponsored search auction to sell these sponsored link placements.
The sponsored search auction was first introduced in 1998 by Goto for Yahoo!.Since then, search engine designers have upgraded the mechanism several times.The purpose of replacing an old sponsored search auction with a new one is "to bring more stability to the auction bidding, increase profits, and help reduce strategic bidding" (Jansen and Mullen, 2008).For search engines, the best scenario is that each of the bidders truthfully spends his maximum willingness to pay on purchasing the links.However, as pointed out by Edelman et al. (2007), bidders have incentive to game the auction system to reduce their costs.In theory, McAdams and Schwarz (2007) argue that the costs that buyers incur while trying to "game" an auction mechanism are fully passed through to the seller.Therefore, search engines may be sensitive to the stability of their sponsored search auction mechanisms.
Correctly understanding how sponsored search auctions work will not only affect the multibillion dollar revenue of search engine companies, but it will also help develop superior sponsored search auctions in the future.In their seminal and influential work, Edelman et al. (2007) and Varian (2007) propose a Generalized Second Price (GSP) auction framework to understand the sponsored search auctions from 2002 through 2007.Edelman et al. (2007) and Varian (2007) argue that this game framework "describes the basic properties of the prices observed in Google's ad auction reasonably accurately."However, Borgers et al. (2007) suggest that "this static GSP auction model actually may have a very poor explanation power on the real data collected from the Yahoo!sponsored search auction".Actually, the GSP auction approach ignores many important aspects of the actual bidding, such as the dynamic bidding nature or the fact that, in practice, bidders often employ automated bidding software to manage their bidding.Therefore, to what extent this GSP framework can explain the actual bidding behaviors still needs careful empirical examination.
This paper reinvestigates this issue by exploiting a natural experiment which took place on June 26, 2002.On that day, Yahoo! switched its sponsored search auction from the Generalized First Price (GFP) auction to the Generalized Second Price (GSP) auction.Auction theory claims that "superior designs" have replaced the "inefficient market institutions" (Edelman et al., 2007 andJansen andMullen, 2008).The GSP game framework predicts that, in the new auction, bidders will be less likely to "game the system".This means that an individual bidder will change his bid less frequently and that his bid range will be smaller.This paper tests these hypotheses using bid data collected from Yahoo! sponsored search auctions in 812 keyword markets from June 15, 2002 through June 14, 2003.
The econometric method adopted in the test is the Regression Discontinuity Design (RDD) approach.Yahoo!'s auction rule change on June 26, 2002 provides a treatment effect framework.Specifically, all the bidders after June 26, 2002 would face the treatment of the GSP auction.Thus, estimating the bidding behavior difference in these two auction systems will be turned into identifying the average treatment effect.However, in the standard treatment effect framework, identification usually depends on strong assumptions.For instance, for the identification of OLS regression, one has to assume that the error terms of the regression satisfy certain ad hoc statistical relationships such as the i.i.d.assumption.This paper avoids this challenge by using a Regression Discontinuity Design (RDD) approach, which enables us to make relatively simple and reasonable assumptions to obtain identification.
The estimation results show that the bidding behavior under the new auction system was less stable than the GSP auction framework predicted.The daily frequency with which an individual bidder changed his bid increased by 3.7 times, representing a 19% increase.In addition, the daily bid range of each bidder increased by $0.71, representing a 46% increase.All the above findings contradict the prediction made by the GSP auction framework which stated that in the new auction bidding behaviors would be more stable.
These findings are important in two aspects.First, in practice, the stability of the sponsored search auction system is an important property.In theory, McAdams and Schwarz (2007) argue that the costs that buyers incur while trying to "game" an auction mechanism are fully passed through to the seller.Therefore, search engines may be sensitive to the stability of their sponsored search auction mechanisms.Actually, in practice, stability has become an important concern.One of the reasons why Yahoo! switched to the GSP auction system was that Yahoo! believed that the GSP auction was more stable and the "second-price structure makes the market less susceptible to gaming" (Edelman et al., 2007).
Second, the findings above have important implications for the current sponsored search auction theory literature.Recent theories on the sponsored search auction, including Edelman et al. (2007), Varian (2007) and Athey and Ellison (2007), are basically based on a static game theory structure.One salient fact these theories miss is that bidders often adopt automated bidding robots to manage their bidding.Yuan (2009) suggests that over 40% of bids may be submitted by these bidding robots following certain technical bidding rules, such as Position Targeting, Cost-per-Purchase Bidding, ROI Bidding, and so on.
It should be noted that this paper is not the first empirical work which provides evidence and suggests that the GSP auction framework may have trouble explaining the actual bidding data.Borgers et al. (2007) suggest that "this static GSP auction model actually may have a very poor explanation power on the real data collected from the Yahoo!sponsored search auction".The difference between this paper and Borgers et al. (2007) is that the latter adopt a structural estimation approach.Plus, to obtain identification, they have to make relatively strong assumptions.Besides, their data set is relatively small.However, this paper exploits the institutional variation and conducts a reduced form regression by employing a RDD approach.This approach enables the author to make relatively simple and clean assumptions.Another advantage this paper has is the huge data set.This paper uses over 1 million bids collected from 812 keyword markets to conduct the RDD estimation, as opposed to the roughly 60,000 bids collected from 5 keyword searches used by Borgers et al. (2007).
The paper is organized as follows.Section 2 introduces the Yahoo!sponsored search auction.Section 3 briefly surveys the sponsored search auction theory literature and proposes the hypothesis to test.Section 4 introduces the data and presents the simple statistics and OLS regression results.Section 5 sets up the RDD framework for the test and discusses the estimation methods.Section 6 presents the RDD estimation results.Section 7 concludes.

Yahoo! Sponsored Search Auction
In the search engine industry, there are three key players: the advertisers, the search engines, and the potential customers.Search engines navigate potential customers to advertisers' product web sites by displaying their web links when potential customers conduct keyword search requests.Advertisers' links are called sponsored links.Sponsored links distinguish themselves from the organic (non-sponsored) web search results by whether or not a fee is paid to the search engine company.Figure 1 shows an example of sponsored links for the key word "refinance".When someone uses Yahoo! to search for information about "refinance", the search engine will display search results along with sponsored links, which are circled in Figure 1.Usually around 10 sponsored links, located on the top and on the right of each page, will be displayed.Advertisers are interested in buying these link slots for their product web sites because they may target the potential customers more efficiently.In 1998, Goto first introduced the sponsored search auction in the search engine industry to sell these link slots.
The sponsored search auction is a multi-object dynamic auction in which each individual advertiser bids for the ideal slot for his web site.Sponsored search auctions usually have the following common features.First, all the link slots are auctioned at the same time.As shown in Figure 1, there were at least 12 sponsored link slots being auctioned at that time.Second, the auction is dynamic with an infinite time horizon.Each bidder can change or withdraw his bid at any time, which will be immediately reflected in his slot placement.Third, all search engines share a common payment rule: pay per click (PPC), which means that whenever there is a click on the sponsored link, the bidder will pay Yahoo! once.And lastly, in Yahoo!'s sponsored search auction, all the information, including bids and slot placement, is public information, which can be observed by all the bidders directly.
In keeping with the keyword search for "Refinance", Figure 2 shows all bidders' bids and slot allocation information as it was captured by a free public web site.This free bid check website is http://keyword.secretstohighpro_t.com/default.aspx.Actually, Figure 1 and Figure 2 were captured at the same time on March 28, 2007.Figure 2 shows that the bid range is from $16.13 to $7.49 and each bidder's position is determined solely by his bid.As can been seen, "Lending Tree" had the highest bid; therefore, this advertisement was placed at the highest slot as shown in Figure 1.
Designing efficient auction rules regarding how advertisers pay a search engine and how a search engine allocates the link slots among its advertisers is a key challenge faced by search engine designers because the decision to adopt different forms of sponsored search auctions has an important impact on the success of search engine companies.In the past six years, Yahoo! upgraded its sponsored search auction several times hoping to find a better auction mechanism to bring more stable bidding behaviors and higher auction revenue.
Before June 26, 2002, a bidder in the Yahoo!sponsored search auction paid Yahoo! his bid multiplied by the number of the clicks on his web site.For example, if a bidder bid $3 and his web site received 3,000 clicks, the bidder would have to pay Yahoo! $9,000.The literature calls this type of sponsored search auction "Generalized First Price (GFP) Auction" to distinguish it from the standard first price auction.
On June 26, 2002, Yahoo! upgraded its Generalized First Price (GFP) Auction to a Generalized Second Price (GSP) Auction.In this new auction system, the web site placement was still determined solely by a bidder's bid, but each bidder, instead of paying his own bid per click, only had to pay 0.01 more than the next highest bid below his.For example, if two bidders bid $0.4 and $0.6, respectively, in the old bidding system, the winner would pay $0.6 per click received; however, in the GSP auction system, he would be charged at a rate of $0.41.

Literature
Recently, economists have been interested in providing a theoretical game foundation for the sponsored search auction between 2002 and 2007.Edelman et al. (2007) first introduced equilibrium concepts for the GSP auctions based on the idea of "envy-free", which assumes that in the equilibrium no bidder would like to place a bid that would cause retaliation.Varian (2007) independently discovered "envy-free" equilibrium and named it "Symmetric Nash Equilibrium" (See Edelman et al. 2007).Many papers follow these two papers and expand the theory framework.In a similar setup, Athey and Ellison (2007) further introduce consumer search behavior into the model and analyze the implications for reserve prices, product variety, and etc. Borgers et al. (2007) make more general assumptions on bidding behaviors and find the multiplicity of Nash Equilibrium.Edelman et al. (2007) and Varian (2007) argue that this game framework "describes the basic properties of the prices observed in Google's ad auction reasonably accurately," Yet Borgers et al. (2007) suggest that "it actually may have a very poor explanation power on the real data collected from the Yahoo!sponsored search auction."This debate suggests that we still need to carefully conduct empirical examinations on the issue.
In the following subsection, this paper will examine the current GSP auction theory and construct testable hypotheses.

The Prediction of the GSP Auction Framework
If the current static GSP auction accurately describes bidders' bidding behavior, the actual bidding should follow the predictions of the theory when it comes to the comparison between the GSP auction and the GFP auction.Currently, theories mainly focus on the GSP auction.In contrast, hardly any formal theoretical analysis has been done on the GFP auction.To compare these two auctions, Edelman et al. (2007) proposed a simple but influential example, which the following literature frequently cited (See Edelman and Ostrovsky, 2006;Jansen and Mullen, 2008).In this subsection, this research also follows this example and constructs the hypothesis.

Example 1 (Edelman et al., 2007):
There are two slots for the links.The first slot receives 400 clicks per hour, and the second slot receives 100 clicks per hour.There are three advertisers bidding to place their product.The value per click for the bidders is $5, $4 and $2.Call these three bidders A, B, and C respectively.Edelman et al. (2007) use this example to illustrate the superiority of the GSP auction over the GFP auction in terms of stability.They show that in the GSP auction, the equilibrium bids of A, B, and C will be $5, $4 and $2 and that with these bids, efficient allocation is achieved.In the GFP auction, however, the equilibrium will not be stable.B will bid $2.01 instead of $4 and A will bid $2.02 instead of $5. B will outbid A at $2.03 and the bids will escalate until $4. B will pull his bid back to $2.01 and the bid escalation will go on again.These bidding behaviors will result in the sawtooth pattern of a bidding war, which is well documented in the literature (Edelman and Ostrovsky, 2006) and (Zhang, 2005).Based on this example, they argue that this second price auction structure makes the GSP auction more stable than the GFP auction.
In the following research, this paper will test the above prediction by estimating how much more stable are the bidding behaviors in the GSP auction.To be specific, if the static GSP auction framework works as the theories claim, the following hypotheses can be tested: Hypothesis 1 On average, an individual bidder in the GSP auction will change his bid less frequently than in the GFP auction.
Hypothesis 2 On average, in the GSP auction an individual bidder's bid range will be smaller than in the GFP auction.

Data
Yahoo!'s research department provides a data set which records all of the bids for the top 812 keyword search by volume and all of the associated accounts for the time period from June 15, 2002 through June 14, 2003.
Each observation in the data contains the following information: bidder ID, his bid, the time when the bid was submitted, and the keyword ID.Both advertisers and keyword phrases are represented as meaningless anonymous numbers so that no identifying information is revealed.The system only records the bid change for 15 minute increments.Therefore, within the 15 minute window, any ordering information cannot be interpreted as chronological.
Table 1 shows several market statistics: the max bid, the mean bid, the minimum bid, and the standard deviation for the top ten most clicked markets.Five cents is the minimum requirement for bidding.One striking observation is the value of the maximum bid.According to this data set, one bidder is paying Yahoo! $9,170 for just one click through the sponsored search.
Table 2 presents the possible evidence of automated bidding behaviors.The first row of the table shows the simple statistics of the number of times each bidder changed his bid each day.On average, among all bidders and all markets, each bidder changed his bid 15 times each day.The maximum number of times a bid changed in one day is 17,867, which is definitely the result of automated bidding.The lower part of the table gives some ideas about the portion of bids submitted by automated bidding software.First, it shows that more than 53% of bidders update bids 40 times each day.In other words, if updating bids 40 times each day implies the employment of automated bidding, around 53% of bids are submitted by bidding robots.Moreover, the table shows that more than 44% of bidders update bids 100 times each day and more than 28% of bidders change their bids over 500 times each day, which is probably the case of automated bidding.To summarize, Table 2 reveals that a significant portion of bids are submitted through automated bidding software.
The prevailing employment of automated bidding software has important implications because the software usually follows certain mechanical rules instead of solving optimization problems.This will raise questions about how accurately the current theories can describe the actual bidding behaviors in the sponsored search auctions.I also present the individual bidding statistics from June 15, 2002 through July 15, 2002 in Table 3. Table 3 provides the maximum value, the mean value, the minimum value, and the standard deviation for the following daily statistics: Bid frequency: the number of times that an individual bidder changes his bid each day.
Bid range: the difference between the maximum bid and the minimum bid of each bidder on each day.

The maximum bid, 75th percentile bid, mean bid, median bid, and 25th percentile bid of each bidder on each day.
Bid frequency and bid range measure the bidding stability of the auction system.The maximum bid, 75th percentile bid, median bid, mean bid and 25th percentile bid measure the impact on the bid distribution of an individual bidder.
Table 3 shows how the mean values of the above statistics change after the launch of the new auction.The mean values of both the daily bid frequency and the daily bid range increase, which suggests that the new auction system is more unstable.The mean values of the max bid and the 75th percentile bid increase while the mean value of the 25th percentile bid decreases, which suggests that the bids are more dispersed.
It is impossible to plot the statistics because of the huge size of the data.To further show a bigger picture of how the change of auction systems affects bidding behaviors, we run a simple OLS regression.In the OLS regression, we control for the market fixed effect and weekday effect.We also control for the entry and exit of bidders by focusing on bidders who submit bids both before and after the auction rule change.
The OLS regression examines the percentage change of the variable y , , in the following three months from July 2002 to September 2002.y , , is the statistic of interest for bidder i, market m, at time t.µ is the market fixed effect.
is an indicator function, showing whether the time t is in the j month after the policy change.Therefore, measures the impact of the new auction system on the market in the first month after the policy change.α measures the impact in the second month after the policy change, and so on.β is the weekday dummy for Tuesday through Sunday.
Table 4 provides the estimation results, showing the bidding behavior change after June 26, 2002.First, both the daily bidding frequency and the daily bid range increase after the auction rule change.This result shows that the bidding behavior is more unstable under the GSP auction in contrast to the predictions which suggest the opposite result.
Second, the individual daily bid distribution expends as the percentage changes of the daily maximum bid, mean bid and median bid are bigger than that of the daily 25th percentile bid.
It may be too ambitious to claim the rejection of the hypotheses based only on the OLS regression results.The unobserved heterogeneities will make these OLS estimations biased, even misleading.First, the OLS regression does not control for the competition brought by the entry and exit of bidders in each market, which is also impossible to do.Fewer numbers of bidders might make the bidding behavior less aggressive.This may bring bias to the estimation of the average daily bidding frequency and the daily bidding range.
Second, the OLS regression cannot control for the competition among search engine markets.During that time period, around 2002, Google's popularity was beginning to rise, becoming more and more popular and grabbing more and more sponsored search market shares.Bidders often had accounts in both search engines.The implication, thus, would be that bidders might have been transferring more resources to Google accounts and paying less attention to their Yahoo!searches.This might also have caused bidders to bid less aggressively, which would affect our OLS estimation.
Moreover, the OLS regression cannot control for many unobserved variables which play an important role in the bidding strategies, such as the bidders' budgets, the conversion rate of purchases, and so on.
The bottom line here is that although the above regressions present a bigger picture of the bidding behavior change and suggest that the GSP auction does not bring stability to the system, the story is not convincing as there are so many unobserved variables which might plague the estimation.Therefore, to identify the true average effect, at least to reveal the sign of it, in the following section we use a RDD approach to avoid the above impacts of the unobserved heterogeneities.

Model
The basic idea behind the estimation is to exploit the natural experiment to identify the treatment effect of the GSP auction, which is the performance difference between the two auction systems.

Let y
, denote the statistics of interest for individual i at time t. is a variable vector including all other characteristics such as market dummy and weekday dummy.
After June 26, 2002, Yahoo! launched a new auction system.Let be the critical time when this new auction started.
Then the statistics y can further be rewritten as and α , captures the impact the new auction rule has on bidders' bidding behavior.is the indicator function of whether the bids were submitted after .There is no restriction on the form of , except continuity at .E α , is the average treatment effect, which is what we want to estimate.
If the hypothesis is valid, E α , should be negative.The challenge is the identification of E α , as discussed in section 4. The unobserved heterogeneities will make OLS estimations biased, even misleading.However, the RDD approach can obtain identification with more relatively simple and clean assumptions.

The Identification by RDD
Following Imbens and Angrist (1994) and Cameron and Trivedi (2005), it is easy to show the following: RDD approach assumes that the bidding statistics under the GFP auction system near the critical value are continuous although we cannot observe these statistics after as the GFP auction was replaced.In other words, bidders would have continued to behave as they would have before the auction rule change as if there had been no auction switch.Therefore, any bidding behavior change will be attributed to the treatment, or the launch of the new auction.Instead of making ad hoc assumptions about the bidding behaviors long before and after and running regressions across all the observations as the OLS method does, the RDD estimation uses nonparametric regressions to only estimate the bidding behaviors shortly before and after , and then compares the behavior difference around to achieve identification.
Because the above continuity assumption is addressing local properties, the estimated result also only reveals the local properties.It is worthwhile pointing out that Regression Discontinuity approach does not identify the average treatment effect E α , of the whole population.Instead, it identifies the local average E α , at , which is E α , | .

RDD Estimator
Imbens and Lemieux (2007) and Van der Klaauw (2007) have very good surveys for the literature of RDD, especially the estimation methods used in RDD.The asymptotical boundary properties of the standard kernel estimator are not ideal because of the poor convergence rate, as pointed out by Hahn et al. (2001) and Porter (2003).Therefore, we consider the local linear regression method proposed by Fan and Gijbels (1996).
Let α and β solve the following minimization problems for the numerator: Here h is the bandwidth on either side of the discontinuity point.is the covariate vector in which the estimation includes the market dummy variable and the weekday dummy variable.
Then the estimator for the average treatment effect will be .

Results
To estimate the bidding behavior difference, we try two different bandwidths on 11 days and 6 days around the discontinuity point on June 26, 2002.In the literature, there are some well-known methods for choosing this bandwidth optimally.Imbens and Lemieux (2007) have very good surveys on the optimal bandwidth selection.Due to the length of data for RDD, which is around 15 days before and after the cutoff point, we pick these two bandwidths.And readers should be noted that the results are not affected by the picks of the bandwidths.The bidding statistics examined here include the daily bid changing frequency, the daily bid range, the daily maximum bid, the 75th percentile bid, the mean bid, the median bid, the 25th percentile bid, and the daily individual bidder payment.For the last statistic, because we cannot observe the click-through-rate of each bidder's link, we simply assume every bidder received one unit of clicks every 15 minutes.
Table 5 shows the RDD estimation results, which are consistent with the OLS regression results shown in Table 4.
Column 1 shows the frequency of individual daily bidding increased 3.7 times, which represents a 19% increase.
For each individual bidder, bid range also increased by 71 cents, or 46% in relative value.All reject the hypothesis made by the static GSP auction framework The estimation results also present how an individual bidder's payment changed each day after the new auction was launched.Table 5 shows that each bidder's average daily payment decreased by about 11.7%.One of the reasons that this number is so big might be that we cannot observe the actually click through rates on each slot and have to calibrate the numbers from Brooks (2005).This might bring bias to the magnitude of the estimation.
Table 5 also presents how an individual bidder's bid distribution changed after the launch of the GSP auction.
Column 2 shows that the maximum bid and the 75th percentile bid tended to increase and the mean bid, median bid and the 25th percentile bid tended to decrease, which is also consistent with the increase of bid range.
Lastly, Figure 3 plots how each of the statistics evolves before and after June 26, 2002.We fit the estimation results with a smooth function.
When Yahoo! launched the new auction mechanism, it named the new bid "maximum willingness to pay", hoping bidders would simply bid their highest possible payment.Yahoo! hoped this would reduce the instability of the system and increase its revenue.But the above results suggest these goals were not achieved.Instead of reducing the strategic behaviors, bidders submitted their bids in a bigger range and changed their bids at a higher frequency.

Robustness Check
The above estimation provides the average effect across the markets.We also conduct the RDD estimation of the individual daily bidding frequency change and daily bid range for each market m. Figure 4 provides the histograms of the RDD estimation results.Most of daily bidding frequency increase resides between 0 and 10 times.The mean and median are also between 0 and 10.Meanwhile, the histogram of the daily bid range increase shows that in most of the markets bidders' increase their daily bid ranges and the mean and median value are around $1. All these results reject the hypothesis and suggest the robustness of the above RDD estimation regarding the daily bidding frequency increase and the daily bid range increase.

Learning
One possible factor, which might have an impact on the above estimation, is the learning curve.If bidders were testing and learning the new auction system, the estimation of the bidding frequency difference may have biases.
There is no good control for learning because of the limitation of the data, especially to what extent learning may affect estimation.However, one important fact is that, before Yahoo!launched the GSP auction in June, Google had already started a similar GSP auction in April (Jansen and Mullen, 2008).Serious advertisers would have usually had accounts with both of the search engines' sponsored search auctions.Therefore, ideally, the bidding behaviors under the Google GSP auction would be a perfect control group for the current RDD analysis.Unfortunately, it is even harder to obtain keyword auction data from both of these two search engines.However, as keyword advertising is a serious business that deals with a serious amount of money, it is reasonable to assume that most of the bidders in the Yahoo!GSP auction had already experienced and learned the GSP auction bidding under Google.Therefore, when they faced the GSP auction under Yahoo!, learning might not play an important role.

Conclusion
The sponsored search auction is an important and interesting mechanism worth great attention.Edelman et al. (2007) and Varian (2007) provide a foundation to understand it.However, the actual bidding in the real sponsored search auction is much more complicated, especially when a significant portion of bids are submitted by automated bidding robots following certain mechanical bidding rules.When one bidder can change his bid over 10 000 times in one a day, an immediate question should be whether the current theory has fully captured the main features of the actual auction and is able to give accurate predictions about bidding behavior.
Fortunately, the institutional variation of the Yahoo!sponsored search auction rule provides an opportunity to examine the theory using a huge data set with over 1 million observations.This paper provides solid evidence suggesting that the GSP auction framework may provide biased predictions on the actual bidding.
Due to the limitations of the data set, it is not the goal of this paper to interpret why the actual bidding does not match the current GSP auction framework predictions.But, we conjecture that the automated bidding in the sponsored search auction may affect the explaining power of the current theory framework.An interesting topic would be to examine what role automated bidding behaviors play in the actual bidding process.This will be left for future research.

Table 1 .
Bid Statistics of the Top 10 Most Clicked Markets

Table 2 .
The Average Bidding Frequency on Each Day

Table 3 .
Summary Statistics from June 15th to July 5th