multiattribute reverse auction
DESCRIPTION
Multi attribute Reverse AuctionTRANSCRIPT
-
Better, Faster, Cheaper: An Experimental Analysis of a Multiattribute Reverse AuctionMechanism with Restricted Information FeedbackAuthor(s): Ching-Hua Chen-Ritzo, Terry P. Harrison, Anthony M. Kwasnica, Douglas J.ThomasSource: Management Science, Vol. 51, No. 12 (Dec., 2005), pp. 1753-1762Published by: INFORMSStable URL: http://www.jstor.org/stable/20110465 .Accessed: 02/05/2011 04:54
Your use of the JSTOR archive indicates your acceptance of JSTOR's Terms and Conditions of Use, available at .http://www.jstor.org/page/info/about/policies/terms.jsp. JSTOR's Terms and Conditions of Use provides, in part, that unlessyou have obtained prior permission, you may not download an entire issue of a journal or multiple copies of articles, and youmay use content in the JSTOR archive only for your personal, non-commercial use.
Please contact the publisher regarding any further use of this work. Publisher contact information may be obtained at .http://www.jstor.org/action/showPublisher?publisherCode=informs. .
Each copy of any part of a JSTOR transmission must contain the same copyright notice that appears on the screen or printedpage of such transmission.
JSTOR is a not-for-profit service that helps scholars, researchers, and students discover, use, and build upon a wide range ofcontent in a trusted digital archive. We use information technology and tools to increase productivity and facilitate new formsof scholarship. For more information about JSTOR, please contact [email protected].
INFORMS is collaborating with JSTOR to digitize, preserve and extend access to Management Science.
http://www.jstor.org
-
MANAGEMENT SCIENCE int!? Vol. 51, No. 12, December 2005, pp. 1753-1762 DOI i0.l287/mnsc.l050.0433 ISSN 0025-19091 EissN 1526-55011051511211753 ? 2005 INFORMS
Better, Faster, Cheaper: An Experimental Analysis of a Multiattribute Reverse Auction Mechanism with
Restricted Information Feedback
Ching-Hua Chen-Ritzo, Terry P. Harrison, Anthony M. Kwasnica, Douglas J. Thomas
Smeal College of Business, The Pennsylvania State University, University Park, Pennsylvania 16802
([email protected], [email protected], [email protected], [email protected]}
The majority of reverse auctions for procurement use a single-attribute (price) format while providing con
straints on nonprice attributes such as quality and lead time. Alternatively, a buyer could choose to conduct
a multiattribute auction where bidders can specify both a price and levels of nonprice attributes. While such an auction may provide higher theoretical utility to the buyer, it is not clear that this theoretical improvement
will be realized given the increased complexity of the auction. In this research, we present an ascending auction
mechanism for a buyer whose utility function is known and dependent on three attributes. Motivated by a
supply chain procurement problem setting, we consider quality and lead time for the two attributes in addition to price. The auction mechanism provides the bidders with restricted feedback regarding the buyer's utility function. We explore, experimentally, the performance of this multiattribute auction mechanism as compared to a price-only auction mechanism. Compared with the price-only auction, we find that our mechanism design is effective in increasing both buyer utility and bidder (supplier) profits. Key words: auctions; experimental economics; supply chain management
History: Accepted by Sunil Chopra, operations and supply chain management; received January 2004. This paper was with the authors 3 months for 2 revisions.
1. Introduction Reverse auctions are auctions in which the auction eer, on behalf of a buyer, solicits bids from a group of potential sellers. They provide a promising alter native to buying goods or services at a fixed price or
through negotiation and can be easily implemented electronically. Covisint, FreeMarkets, General Electric (Kwasnica and Thomas 2002), and Mars (Hohner et al.
2003) are examples of firms that have attempted to use auctions for this purpose. While auctions are an excellent form of price discovery, there are usually other aspects in addition to price that can affect the value of the outcome. In such cases, if the auction
process focuses competition on price only, the buyer is forced to ignore these other aspects or to handle them outside of the auction by subsequent contractual
arrangements.
Presumably, the buyer would like to negotiate all dimensions of the contract with all potential sellers to discover the most valuable trade. However, it is
not clear whether a multiattribute auction mechanism can be designed that effectively achieves this, in the
ory and in practice. A multiattribute auction mecha nism should possess the following two characteristics: First, the buyer must be able to assess the value of a
multiattribute bid. Second, a bidder must be able to
bid on several attributes simultaneously. These chal
lenges raise two important questions: Is the multiat tribute auction superior to the price-only auction for the buyer/seller in theory? What about in practice? In this paper, we address these questions for a spe cific multiattribute auction mechanism that provides restricted information feedback to the bidders.
A number of researchers have considered dimen
sions other than price in the procurement setting from a purely theoretical perspective. Che (1993) and Branco (1997) study the optimal (buyer utility-maxi
mizing) auction when there are only two attributes, price and quality. In their models, however, bid der private information can still be represented by a one-dimensional type. This simplification allows them to apply powerful mechanism design tools first discussed by Myerson (1981). While there has been some recent promising work on multidimensional
mechanism design (Armstrong 2000, Williams 1999), the extension of one-dimensional optimal auctions to
higher dimensions is not straightforward. Recently, Parkes and Kalagnanam (2002), Beil and Wein (2003), and Asker and Cantillon (2004) have studied auctions
with the potential for many attributes. Parkes and
Kalagnanam (2002) look for efficient (total surplus maximizing) auctions when there may be many
1753
-
Chen-Ritzo et al.: A Multiattribute Reverse Auction Mechanism with Restricted Information Feedback 1754 Management Science 51(12), pp. 1753-1762, ?2005 INFORMS
attributes and products. Beil and Wein (2003) focus on a procurement process similar to ours, but examine a mechanism by which the buyer can learn the sup pliers' cost functions when suppliers are assumed to bid myopically. Asker and Cantillon (2004) study the general properties of multiattribute auctions when the
buyer's scoring function is linear in price. However, even strong theoretical properties may
not be sufficient to justify the widespread applica tion of an auction. Vickrey sealed-bid auctions, for
example, are rarely used in practice despite their attractive theoretical properties (Rothkopf et al. 1990). The English auction, on the other hand, is ubiqui tous.1 Numerous experimental studies have shown
that, even in simple price-only auctions, actual human behavior may differ systematically from what theory predicts (Kagel et al. 1987, Coppinger et al. 1980).2 Because experiments may provide a close parallel with actual auction performance, the laboratory has been successfully used to testbed auction designs in
complex settings (Banks et al. 1989, Brewer and Plott 1996, Kwasnica et al. 2005).
For the reasons given above, we turned to the
experimental laboratory to examine our multiattribute auction design. Our goal was to design an auc tion that, much like the traditional English auction,
not only had good theoretical properties, but also
performed well experimentally. Also, because both bidder (seller) and buyer behavior may matter in the overall performance of the auction, we isolated the performance of the auction with respect to bid der behavior. With this in mind, we developed and tested an auction design that explicitly values three
attributes?price, quality, and lead time?and can sig nal bidders on all three attributes simultaneously. Based on laboratory experiments with human sub
jects, the results are promising. In addition to hav ing desirable theoretical performance, our mechanism leads to increased buyer utility while maintaining seller profits relative to a price-only auction. We do not claim, however, that our auction design is opti mal. Rather, our contribution is to propose a the
oretically sound multiattribute auction design and show its viability through the use of human-subjects experiments.
We highlight the following three characteristics of this research. First, we do not fully reveal the
buyer's utility function. Second, we compare, at both the aggregate and individual level, observed bidder
behavior with the theoretically expected behavior in both a price-only and a three-attribute auction. Third,
our comparison of the single-attribute English auction to the three-attribute English auction is motivated by a
specific supply chain procurement setting. A differentiating feature of our auction design is
the type of information we provide the bidders about the buyer's utility function. Instead of revealing the
buyer's entire utility function, we only provide infor mation on the marginal utilities related to submitted bids. This information can be used to guide bidders toward their optimal bid. We made this design choice for two reasons. First, in many applications it may not be practical for a buyer to reveal their entire util
ity function, either for strategic reasons, or because
they are unable to express it. Second, the informa tion provided to the bidders is designed to be simple and intuitive. Previous experimental studies of com
plex auctions have demonstrated that simple infor mation can often be effective in generating efficient outcomes; Kwasnica et al. (2005) found that properly designed single-item prices in a combinatorial auc tion can improve efficiency despite the fact that, in
theory, these prices do not provide all the necessary information. It is possible that the limited informa tion we provide may actually improve auction per formance relative to fully revealing the buyer's utility function by simplifying the situation for the bidder.3
One of the objectives of this research is to investigate whether a reasonably successful multiattribute auc tion can be designed with information that is both
readily available for the buyer and simple for the bid ders to understand.
Bichler (2000) was one of the first to use experi ments to study multiattribute auctions. He examined
two-attribute auctions using human subjects experi ments and found that the auctions, in general, have modest gains over single-attribute auctions. It is pos sible that, due to the added complexity of the auc
tion, the gains observed with only two attributes
may disappear when more attributes are considered. Therefore, in our experiments, we consider an auc
tion where the buyer and bidders care about three attributes (i.e., price, quality, and lead time). In his experiments, Bichler revealed the entire scoring func tion used by the buyer. Strecker (2003) conducted three-attribute English auction experiments, where the buyer's scoring function was not at all revealed in one treatment and fully revealed in the other. In contrast to both these studies, we examine a multiat tribute auction that provides partial revelation of the
buyer's scoring function.
1 Cramton (1998) discusses the pros and cons of using an English auction rather than a sealed-bid mechanism. 2 See Kagel and Roth (1995) for a good, albeit somewhat outdated, review of the large experimental literature on simple auctions.
3 For example, if we published the buyer's utility function, bid ders would need to perform calculations to arrive at the optimal attribute bid levels. Assuming the bidders recognize that they need to do this, it is still questionable whether they would make the correct calculations.
-
Chen-Ritzo et al.: A Multiattribute Reverse Auction Mechanism with Restricted Information Feedback
Management Science 51(12), pp. 1753-1762, ?2005 INFORMS 1755
Both Strecker (2003) and Bichler (2000) provided bidders with tools for computing the buyer's util
ity for all possible bids, whereas the issue of what kind of information to provide bidders within a three attribute auction setting was studied by Koppius and van Heck (2003). They designed an experiment in
which they exposed subjects to different information architectures?a restricted architecture and an unre stricted architecture. In the restricted architecture, bid ders were informed of the current best bid and bidder. In the unrestricted architecture, they were also pro vided with a ranking of the most recent losing bids.
Koppius and van Heck (2003) found that both the effi ciency and Pareto optimality are greater in the unre stricted information architecture case. Although their
experiment addressed the effect of two specific types of information exchange on the aggregate outcomes of the multiattribute auction, they did not compare these results against the single-attribute auction case.
Moreover, the information presented to the bidders in both cases is actually very limited with respect to
guiding bidders toward their optimal bid. We elabo rate on this point in ?3.1.
This paper is organized as follows: In ?2, we describe and motivate the functional forms for buyer utility and bidder profit functions. In ?3, we present the auctions examined. In ?4, we describe the exper imental design. Results are presented in ?5. We con clude our work and discuss future directions in ?6.
2. The Environment There is a single buyer who is procuring a single indi visible object or contract from a set of three potential suppliers.4 The contract has three measured attributes of interest: price p, quality level q, and lead time Z. Lead time is typically thought of as the amount of time that the buyer must wait between order and
receipt of goods. Throughout, we use the subscripts 0 for the buyer and i for the bidders and the sub
scripts Q and L to denote parameters related to qual ity and lead time.
The buyer's utility function for the contract is linear in price, with v(q, I) representing the value of the two
nonprice attributes quality q and lead time I. In our
experiments, the buyer's utility function for a contract is given by
U(p, q, I) = v(q, l)-p = aQOqb& + aL0(l -
l**) -p, (1)
where aQ0 > 0 and bQ0 are quality-related constants and aL0 > 0 and bL0 are lead-time-related constants. The buyer always prefers higher quality. In our exper iments, we chose bQ0 such that the marginal value of quality is decreasing (0 < bQ0 < 1).
The profit function for bidder (supplier) i for an awarded contract is linear in price, with c{(q, I) rep resenting bidder i's cost as a function of quality and lead-time levels. In our experiments, bidders have cost functions that yield the following profit function:
ir,-(P, (2)
where aQi > 0 and bQi are quality-related constants and aLi > 0 and bLi are lead-time-related constants. Each bidder's cost increases with improvements in both quality and lead time. The marginal cost of
higher quality is increasing (bQi > 1). If the bidder does not win the auction, she will have a profit of zero.
Define the total welfare created by a contract between the buyer and bidder i at quality and lead time levels (q, I) by
WK?, /) = U{p, q, I) + fli(p, q, 1) = v(q, I) - Ci(q, I). (3) Let (q*, I*) be quality and lead-time levels that max imize Equation (3). Note that the buyer can select at
most one supplier, and that total welfare is unaffected
by the contract price. Thus, an efficient procurement contract is given by the supplier (i) and the corre sponding (q*, I*) that maximize total welfare.
3. The Auctions We consider two auction mechanisms. Both auctions
are generalizations of the standard reverse English auction to the multiattribute setting. The first is the
multiattribute (MA) auction that allows for bidding in all three attributes. We compare this auction to the price-only (PO) auction where reservation qual ity and lead-time levels are predetermined and bids are
only accepted on price. Only integer-valued prices are accepted, and quality and lead-time offers are
required to be integer values ranging between 1 and 10. The bidders were human subjects, whereas the buyer was represented by the computer. Other than a time constraint, bidders are free to place any
bid at any time. Before the auction ends, the provi sionally winning, or standing, bid is the one that max imizes the buyer's utility among all the bids placed since the start of the auction.5 Once a bid is placed, it cannot be cancelled.
For both the PO and MA auctions, a "soft" stopping rule was used. That is, after an initial time period, the
4 We recognize that there may be interesting questions related to
the incentive to select multiple suppliers at potentially different attribute levels. Bichler and Kalagnanam (2005) consider multi
sourcing for multiattribute auctions. We focus on the design of the auction when the buyer must select at most one supplier and the
suppliers can offer different levels of attributes. 5 In the event of tie bids, the bid placed earliest always wins.
-
Chen-Ritzo et al.: A Multiattribute Reverse Auction Mechanism with Restricted Information Feedback 1756 Management Science 51(12), pp. 1753-1762, ?2005 INFORMS
auction ended if no new standing bids were placed for k seconds. The purpose of extending the auction in this way was to eliminate the advantage that might
be provided to bidders by sniping (i.e., placing a bid in the last few seconds of the auction).6 In the PO auc tion, the initial time period was two minutes and /c = 45 seconds. In the MA auction, the initial time
period was five minutes and k = 90 seconds.
3.1. The Multiattribute Auction In the MA auction, bidders were required to iden
tify multiattribute bids that were not only profitable, but that would also improve the buyer's utility with
respect to the current standing bid. A critical element of the MA auction is the feedback provided to the bidders. Because we have in mind future applications where the buyer utility function may not be known with certainty, we opted against directly publishing the buyer's utility function. Instead, after each bid, simple information about the buyer's preferences was revealed to the bidder. First, the standing bid was
always displayed. Second, the bidder was given three
pieces of information related to the bid she placed: the price difference (PD), the marginal quality (MQ), and the marginal lead time (ML). The price differ ence is the amount that the bidder must lower her bid in the price attribute for it to become a winning bid
(assuming no changes in quality and lead-time lev els). The marginal quality and lead time are simply the buyer's marginal utility for improvements in q or Z at the bidder's most recently placed bid. Importantly, in our experiments, the MQ value combined with the
bidder's marginal cost provided her with sufficient information to eventually bid at q*. Similarly, ML and
marginal cost information will direct the bidder to /*. Both the information structures studied by Koppius and van Heck (2003) do not necessarily have this fea ture. Under Koppius and van Heck's restricted infor
mation condition, the bidder only receives feedback on the winning bid; this suggests how to beat the current bid but not how to find the optimal bid for each particular bidder. Under the unrestricted con
dition, the bidder observes more information in the form of the relative ranking of the bids. However, the usefulness of this information in supporting opti
mal bidding behavior depends on the actions of the other bidders.7 An advantage of our information feed back is that the bidder can independently determine her unique optimal attribute combination. It remains to be seen whether this information can be properly
used by the bidders; that issue necessitates our experi mental examination. Wfhile the above feedback is pro vided for every bid submitted in the MA auction, throughout the auction, only bids that beat the cur rent best bid are accepted.
When bidders can determine the optimal quality and lead-time combinations (q*,l*), it is straightfor
ward to show that it is a dominant strategy for them to bid at those levels. Therefore, consistent with the standard English auction, it is also a dominant strat
egy to continue lowering the price offered until they make zero profits. If all bidders follow their dominant
strategy, the auction will be efficient and the expected utility of the buyer will be equivalent to the second
highest welfare offer of the bidders. See Chen-Ritzo et al. (2005) for a formal discussion of these properties of the MA auction.
If the buyer had information about bidder costs, then he may find it in his interest to misrepresent his true utility function in the MA auction. Che (1993) shows that this is optimal in a simpler multiattribute
setting where bidder types are still one-dimensional. Of course, any misrepresentation will result in lost
efficiency. We did not examine buyer misrepresenta tion because our focus is on examining bidder behav ior in these auctions. Specifically, the focus of this
paper is on whether an auction can be designed that
provides sufficient decision support to allow the bid ders to bid as theory predicts in a complex, multi attribute setting. The human-subject bidders in our experiment knew that the utility or scoring func tion used by the auctioneer was programmed by the
experimenter. Therefore, issues of buyer misrepresen tation were not a concern.
3.2. The Price-Only Auction In the PO auction, all bids are simply a state
ment of a price at which the bidder is willing to
supply the product. Before the auction begins, the
buyer announces reservation levels for the nonprice attributes. Let qr and V be the reservations levels. All bids are then evaluated as if they were placed with qr and V in the other attributes. Bidders were expected to identify the least cost quality and lead-time combi
nation satisfying these reservation levels to determine their reservation price.
Given the properties of the bidders' cost functions, bidders would never voluntarily deliver the product at a quality or lead time better than the reserve levels.
Once the reserve levels are announced, the PO auction is a standard reverse English auction. It is a domi
nant strategy for each bidder to bid down to the point where, at the reserve levels, profits are zero, mean
ing that the auction will end at the second-lowest cost given the reserve levels. Because the reserve levels eliminate many bid choices for the bidders,
6 See Roth and Ockenfels (2002) for more on the strategy of sniping. 7 As an example, imagine the extreme condition where all bidders
placed bids with identical nonprice attributes. In that case, the rank
ing would provide no more information about potential improve ments than a simple price-only auction.
-
Chen-Ritzo et al.: A Multiattribute Reverse Auction Mechanism with Restricted Information Feedback
Management Science 51(12), pp. 1753-1762, ?2005 INFORMS 1757
it is straightforward that buyer utility will always be at least as high in the MA auction as it is in the PO auction (Chen-Ritzo et al. 2005).
To give the PO auction the best chance of suc
cess, we chose the reserve levels for lead time and
quality that maximized buyer utility assuming full
knowledge of the bidder profit functions. While it is unrealistic to expect that a buyer would have full information when setting the reserve levels, we selected this standard to provide a difficult test for the MA auction with which to compete. It is worth
while to note that the reserve levels (for quality and lead time) that maximize buyer utility are not neces sarily the efficient levels. Furthermore, even though total welfare in the MA auction is no smaller than
welfare in any PO auction, the profit of the winning bidder may be lower in the MA auction.
4. Design of the Experiment We considered four different treatments in our exper
iment: (1) The PO auction with inexperienced bidders (PO), (2) the PO auction with experienced bidders (P02), (3) the MA auction with inexperienced bid
ders (MA), and (4) the MA auction with experienced bidders (MA2). The response variables of interest were, primarily, the realized buyer utility and seller
profits. We recruited subjects for a total of 24 experi
mental sessions,8 where in each session we exposed subjects to one of the four treatments previously described. Each session consisted of several indepen dent auctions, or periods. The same three bidders
were matched with each other in all periods, and each
group of bidders participated in either the PO auc tion or the MA auction for all periods. We decided to study the auction institutions in separate sessions rather than using a within-subjects design where the same subjects would have participated in both the PO and MA auctions. This allowed us to isolate the
learning effects that we expected to observe for a
given auction type from potential order effects (i.e., variation due to the order in which subjects would have participated in the different auction types in a
within-subject treatment). Additionally, because our subject pool was relatively homogeneous, it is reason able to assume that any effects relating to differences between the subject groups would be small.
In every auction (or period), subjects were provided with information about their bid history in the cur rent auction, their costs for delivering the asset at all
possible lead-time and quality combinations, the time
remaining in the auction, the current best bid and their bid status (i.e., whether or not they were the
winning bidder). The costs of the bidders were dif ferent from period to period and were different from
bidder to bidder. The buyer's utility function was also different from period to period. To help bidders iden
tify profitable bids, they were provided with a tool built into the computer interface that would compute the profit associated with a particular bid.
A total of nine auctions (or periods) were com pleted in each experimental session, the first of those being a practice period for which subjects
were not paid any earnings. Each of four different cases (labeled Case 1 through Case 4) was repeated once during the eight nonpractice periods. A dif ferent case from these four was used in the prac tice period. The sequence of cases was kept constant across all sessions so as not to affect comparisons of results across sessions. Whenever a case was repeated
within a session, the matching of subjects to bid der types was changed so that no subject was the same type in both replicates of each case. This elim inated learning effects that might be associated with
playing the same bidder type more than once. The
subject-bidder type matchings were kept constant across all sessions. The exact parameters used in each case, experiment instructions, screen shots, and other information are available in the online supplement (http://mansci.pubs.informs.org/ecompanion.html).
All experimental sessions were conducted at the Smeal College of Business's Laboratory for Economic
Management and Auctions (LEMA), The Pennsylva nia State University, between March 2003 and April 2003. Any undergraduate or graduate student who had not previously participated in this experiment was considered to be eligible for treatments that
required inexperienced bidders. Only those students who had previously participated in the PO auc tion (MA auction) were eligible to participate in the PO auction (MA auction) treatments that required experienced bidders. The auctions were implemented using Z-Tree software (Fischbacher 1999).
In the PO auction sessions, each period lasted
approximately 5-10 minutes, and in the MA auc tion sessions, each period lasted approximately 10-15
minutes. The PO auction sessions took around 1-1.5 hours to complete, while the MA auction sessions took around 1.5-2 hours to complete, on average. The
exchange rate used in all treatments was 0.1 U.S. dol lars for every experimental dollar. A $7 show-up fee
was added to the total earnings of each subject at the end of the session. No incentive besides cash
was provided to subjects. Subjects earned, on aver age, $22 and $16 in the MA and PO auction sessions, respectively.
5. Results In this section, we compare the performance of the
MA auction and the PO auction for bidders with and 8 A session refers to a continuous period of time where a fixed
group of subjects is exposed to a single experimental treatment.
-
1758 Chen-Ritzo et al.: A Multiattribute Reverse Auction Mechanism with Restricted Information Feedback
Management Science 51(12), pp. 1753-1762, ?2005 INFORMS
Table 1 Expected and Observed Outcomes for Each Case and Treatment Condition
Case 1 Case 2 Case 3 Case 4
Expected MA (PO) outcome Price 130(550) 52(299) 561(294) 2,399(2,364)
Quality (reserve) 4(4) 3(2) 3(4) 10(9) Lead time (reserve) 10(5) 10(6) 3(7) 5(3)
Winner 3(2) 3(2) 2(3) 1(1) Buyer utility 285 (250) 98 (84) 333 (237) 1,986 (1,870) Seller profit 62(10) 46(4) 48(6) 88(49) Improvement in buyer utility 14% 17% 41% 6%
MA (PO) observed Average price 646.6 (559.5) 190.7 (297.2) 577.1 (381.9) 2,035.5 (2,350.6) % Expected quality 44% (N/A) 38% (N/A) 38% (N/A) 50% (N/A) % Expected lead time 25% (N/A) 56% (N/A) 44% (N/A) 36% (N/A) Average distance 4.33 (N/A) 2.45 (N/A) 2.52 (N/A) 2.45 (N/A) % Expected winner 25% (80%) 63% (70%) 75% (85%) 50% (45%) Average buyer utility 240.5 (241.5) 98.5 (85.9) 281.3 (249.1 ) 1,887.1 (1,884.4) Average seller profit 44.1 (-128.4) -6.1 (-64.6) 2.6 (-25.2) 115.4 (-580.5) Number of observations 16 (20) 15 (20) 16 (20) 14 (20)
MA2 (P02) observed Average price 126.7(630.2) 94.5(299.8) 595.3(393.8) 2,090.0(2,355.2) % Expected quality 100% (N/A) 33% (N/A) 67% (N/A) 67% (N/A) % Expected lead time 100% (N/A) 100% (N/A) 67% (N/A) 33% (N/A) Average distance 0.00 (N/A) 0.67 (N/A) 0.47 (N/A) 1.69 (N/A) % Expected winner 100% (83%) 100% (100%) 100% (100%) 67% (50%) Average buyer utility 288.3 (170.8) 94.2 (83.2) 300.7 (237.2) 1,863.7 (1,879.8) Average seller profit 58.1(87.6) 46.8(4.0) 62.0(5.8) 149.0(39.6) Number of observations 6(6) 6(6) 6(6) 6(6)
without previous experience. In Table 1, we present the expected outcome (i.e., the winning bid, win
ning bidder, buyer utility, and seller profit) and the observed average outcome for both auction institu tions, in each of the cases and treatments studied. The
number of observations collected for each treatment in the different institutions is also included in this table.9 Table 2 presents the results of the rank-sum tests that we use for statistically comparing certain outcomes between treatments.10
We begin by examining the utility that the buyer achieves at the end of the auction. In theory, because bidders (sellers) have more choice in their bidding under the MA auction, they should provide the buyer with higher utility. However, the added complexity of the MA auction may keep the buyer from fully real
izing these potential gains. Conclusion 1. The MA auction increases buyer
utility. In Table 1 we report the expected improvement
in buyer utility in the MA auction as compared to
the PO auction. Based on the expected improvements reported in Table 1 and the results reported for the first two sets of rank-sum tests in Table 2, the MA auc tion does result in significantly greater buyer utility
when the expected improvement from the MA auc tion is large (17% or greater). With both inexperienced and experienced bidders, we can reject the equal ity of the observed buyer utility, and conclude that the MA auction provided stochastically greater buyer utility in Cases 2 and 3. Case 1 has a somewhat lower
expected improvement (14%), and we find that buyer utility is greater from the MA auction only under
Table 2 Wilcoxon-Mann-Whitney Rank-Sum Test Results
Test(fy,) Casel Case 2 Case 3 Case 4
Buyer utility w = 325.0 w = 343.0 tv = 368.0 w = 272.0 MA>P0 p = 0.1802 p = 0.0682 p = 0.0112 p = 0.1764 Buyer utility w = 57.0 w = 53.5 iv = 51.0 w = 39.0
MA2>P02 p = 0.0023 p = 0.0079 p = 0.0311 p = 0.5000 Seller profit n/ = 262.0 w = 246.5 w = 355.0 w = 385.0
MA^PO p = 0.2858 p = 0.4414 p = 0.0616 p = 0.0000 Seller profit iv = 33.0 iv = 51.0 u/ = 52.0 w = 28.5
MA2^P02 p = 0.3751 p = 0.0557 p = 0.0435 p = 0.1025 Euclidean distance w = 220.0 w = 202.0 iv = 210.0 iv = 157.0
M/4>M/12 p = 0.0038 p = 0.0883 p = 0.0264 p = 0.2139
Note. The left-hand column represents the measured variable and the alter native hypothesis of the statistical test, w is the rank-sum test statistic and
p is the p-value.
9 The number of observations vary in the MA treatment due to a technical difficultly related to the recording of the results to a data file. 10 The Wilcoxon-Mann-Whitney rank-sum test is a powerful non
parametric substitute to the standard f-test when data has at least ordinal measurement (Siegel and Castellan 1988). When examining data generated from human subjects, it is typical to assume that the data do not meet the assumptions required for a f-test.
-
Chen-Rltzo et al.: A Multiattribute Reverse Auction Mechanism with Restricted Information Feedback
Management Science 51(12), pp. 1753-1762, ?2005 INFORMS 1759
the experienced treatment. Case 4, on the other hand, has only a small expected difference between the MA and PO auctions at 6%. As the toughest test for the
MA auction, it is not surprising that we are never able to conclude that buyer utility is greater under the MA auction. The PO auction does not provide the
buyer with greater utility in any case.
Although we have seen that the MA auction can
significantly improve buyer utility, it is worthwhile to examine how well the auctions performed relative to the expected outcomes. Again, referring to Table 1, we find that on average, the MA auction achieved 91% and 95% of the expected buyer utility for the inexpe rienced and experienced treatments, respectively. The PO auction tends to yield slightly higher utility than
predicted. On average, the PO auction yields 101% of the buyer utility predicted. This difference disap pears in the experienced treatment, and the P02 auc tion only yields 92% of the predicted buyer utility.
The auctions' performances relative to the theoretical benchmarks may shed light on the fact that we did not always observe a significant increase in buyer util
ity in the MA auction relative to the PO auction. At least for the inexperienced treatments, the MA auction fails to achieve full expected buyer utility, whereas the PO auction tends to overshoot predictions slightly.
The winning bidder, quality, and lead time are fre
quently as predicted for the MA auction format. As an additional measure of the closeness of the observed
multiattribute bid to the expected bid, we report (Table 1) the Euclidean distance of the observed qual ity and lead-time levels from those expected in the
MA auction. With the exception of Case 1 for the MA treatment, this distance is usually small.
The MA auction also offers potential gains for the bidders. In all four cases, the theoretical seller profits are higher under the MA auction than the PO auction.
Although this may seem to contradict the previous result, the added flexibility of the MA auction often benefits both parties. We examine seller profitabil ity under the different auction institutions in Conclu sions 2 and 3.
Conclusion 2. The MA auction does not degrade, and occasionally increases, seller profits.
The third and fourth sets of rank-sum tests in Table 2 compare seller profits in the MA and PO auc tions. In no case can we conclude that seller profit is
higher under the PO auction. In some cases, 3 and 4 for inexperienced bidders and 2 and 3 for experienced bidders, seller profits actually increase significantly under the MA auction. These observations are con sistent with the theoretical predictions; for the cases studied the theory predicts higher seller profits. We did not study cases where the theory predicts lower seller profits.
As Table 1 shows, average seller profits are often
negative. This is because, occasionally, sellers placed bids that were not individually rational. In 11 out of 80 (14%) PO observations and 5 out of 62 (8%)
MA observations, the winning bidder ended up los
ing money. While it is impossible to determine the source of this behavior, bidders were not in any way forced to bid below their cost (at a loss) and were given checks to help avoid this behavior. We feel the most likely determinant of these losses was the
complex setting the bidders had to understand in a time-constrained, competitive environment, which
was our primary motivation for studying these auc tions experimentally.
Conclusion 3. The frequency of seller losses are not related to the auction institution.
Despite the somewhat greater prevalence of bid der losses in the PO auctions, there is no statisti
cally significant difference between the proportion of observations with losses under the different institu tions. A chi-squared test for differences in proportions yields a test statistic of 0.5807, which is not signifi cant at any reasonable level. The fact that there is little difference in the observed frequency of losses under the two institutions is actually somewhat surprising.
A priori we expected that bidder errors would be a
greater problem in the MA auction. In principal, the PO auction was no different from a standard reverse auction, where previous experiments rarely observed bidder losses. However, it appears that the added information of the multiattribute cost tables, despite its irrelevancy in the bidding, might have affected
bidder behavior. Consistent with other experimen tal studies of auctions, we have identified behavior that differs dramatically from the theory. The level of losses actually tends to be greater in the PO auction as well; losses averaged 1,641 experimental dollars in the PO auction versus 217 experimental dollars in the MA auction. Using a rank-sum test, we find that losses under the PO auction are stochastically larger (w = 57.0 and p < 0.0200). These high losses also affect the buyer utility results presented earlier because bid der losses tend to manifest themselves as increased
utility for the buyer. When observations with losses are removed, we find that the PO auction buyer utility as a percentage of expected drops from 101% to 93%. The decline in the MA auction is more modest from 91% to 88%. All buyer rank-sum utility comparisons presented earlier in support of Conclusion 1 still hold if observations with losses are dropped.
Experienced bidders never made losses in either the P02 or MA2 treatments. Therefore, it appears that losses might be more related to learning the interface and environment rather than some systematic prob lem with the auction setup. This also suggests our
next conclusion.
-
Chen-Ritzo et al.: A Multiattribute Reverse Auction Mechanism with Restricted Information Feedback 1760 Management Science 51(12), pp. 1753-1762, ?2005 INFORMS
Conclusion 4. Performance of the MA auction institution improves with experience.
While buyer utility and seller profits rarely change significantly from the inexperienced to the experi enced treatments (see Table 2), the auctions appear to perform more consistently with expectations when there are experienced bidders. When comparing the
MA and MA2 treatments, the proportion of times the predicted winner is observed increases from 53% to 92%. The proportion of correctly predicted qual ity and lead times increases from 42% to 67% and from 40% to 75%, respectively. These differences in
proportions are all significant at the 10% level; a chi
squared test comparing these proportions yielded test statistics of 9.485 (winner), 3.303 (quality), and 6.993 (lead time). If we measure the Euclidean distance
of the winning quality and lead-time combination from the expected outcome, we find that it decreases from 2.96 to 0.71 on average. In three of four cases, the distance drops significantly from inexperienced to
experienced treatments (see Table 2). As a result, as mentioned earlier, the observed buyer utility as a per centage of the predicted increases from 91% to 95%
with experience. The performance of the PO auction improves less
dramatically. While losses are no longer observed and bidders always provide the reservation quality and lead time,11 the proportion of times that the predicted
bidder wins only increases from 70% to 83%, which is not statistically significant (chi-squared test statis tic of 1.059). In addition, buyer utility actually goes down relative to predicted utility; buyer utility as a
percentage of predicted drops from 101% to 93%. The previous results have highlighted the aggre
gate performance of the MA and PO auctions. We can also examine individual bidding behavior and how it relates to the theoretical predictions. Bidders should
place bids at the optimal levels of quality and lead time (??*,/*) in the MA auction. Only 9% and 11% of all bids placed in the MA and MA2 treatments,
respectively, are at the predicted levels. This is not
particularly surprising once one considers the fact that bidders must bid to learn the buyer's utility func tion in the experiments. However, we would expect that bids near the end of the auction will be close to the optimal levels. When considering only the best
(highest utility) bid placed by each bidder, the propor tion of optimal bids increases to 19% (MA) and 31% (MA2). This difference in proportions is significant at the 10% level (chi-squared test statistic of 3.26), indicating that experience increases the incidence of
behavior consistent with the theory. Although this result tells us that bidding behavior, even in the
experienced treatment, is often different than the the
ory predicts, it may be that bidders are only mak
ing small errors in their bidding. Given the struc ture of the cost and utility functions utilized, there
may be only small gains from movements to the effi cient quality and lead-time levels. By measuring the Euclidean distance of each bidder's best bid from the predicted quality and lead-time levels, we have a
measure of distance from the predicted strategy. In the MA and MA2 treatments, the distance averaged 2.40 and 2.01, respectively. Even more telling is that only 3 out of 33 total bidders averaged bids within one unit distance of the optimal strategy.
Even though bidders are not always bidding the correct levels of quality and lead time, they may still be placing bids close, in utility terms, to the opti
mal bid. In theory, losing bidders should offer a bid that provides all the surplus, at the efficient quality and lead time, to the buyer. By examining the highest utility offers of the losing bidders, we can examine the
accuracy of this benchmark.12 We can examine when
losing bidders leave "money on the table," or they do not bid despite the fact that there was a profitable bid for them that would beat the final winning bid. In the
inexperienced treatment, 51% of losing bidder obser vations (there are two observations for each auction period) failed to bid when a profitable bid was avail able. The proportion falls to a more reasonable, but still large, 35% in the experienced treatment. A chi
squared test for difference in these proportions yields a test statistic of 2.70, which is significant at the 10% level. This suggests, along with the previous result, that bidders are often failing to realize the optimal
bidding strategy. This is in contrast to the relatively strong performance of the MA auction on an aggre gate level. To reconcile these results, it must be that the actual utility offered by these bidders is close to the theoretical prediction. We can measure this by cal
culating the utility of each losing bidder's best bid
(i.e., the bid that provided the buyer with the greatest utility) as a percentage of the maximal possible util ity that bidder could offer (i.e., provision of greater
buyer utility would result in bidder losses). Figure 1 shows the fraction of the maximum possible buyer utility that was offered by each losing bidder's best bid in the MA auctions.13 In the MA and MA2 treat
ments, respectively, the losing bidders offer 57% and
11 In the inexperienced treatment, three bidders provided either
quality and/or lead time in excess of the reservation amounts.
12 The bids of the winning bidders cannot be examined because they should stop bidding when others stop bidding. Thus, they will not
bid up to their zero profit point. 13 It is important to note that this measure is biased downward.
Even if a losing bidder is following the optimal strategy, she may
appear to be offering less than 100% under this measure. This is because the winning bidder may jump over the maximal utility offer of the losing bidder, making placement of such a bid unnec
essary. We decided to use this measure because it gives noncom
pliance with the theory the benefit of the doubt.
-
Chen-Ritzo et al.: A Multiattribute Reverse Auction Mechanism with Restricted Information Feedback Management Science 51(12), pp. 1753-1762, ?2005 INFORMS 1761
Figure 1 Fraction of Maximum Available Buyer Utility Offered by Losing Bids Only
1.0
0.8
?0.6
CO
0.2
0
x x\ Xxxx ***?**. xX xxx
x
x
X XX i-Xa )X?3? i-r^X-rX-Xr
X MA MA2
12 3 4 5
Period
78% of the maximal possible utility on average. Using a rank-sum test, we find that losing bidders leave sig nificantly more money on the table in the inexperi enced case than in the experienced case (w = 348 and p < 0.016).
Conclusion 5. Individual bidding behavior in the MA auction is different than the theory predicts, but
improves with experience. Given previous experimental results on bidder
behavior in auctions, it should not be entirely sur
prising that specific theoretical predictions for the MA auction do not do particularly well. Rather, a more appropriate comparison would be to the indi vidual bidding behavior in the PO auction. In the PO auction, theory predicts that losing bidders should bid up to their zero profit condition at the reserve
quality and lead time. Using the same standard as
presented earlier for the MA auction, we find that 31%
(PO) and 39% (P02) of all losing bidders left money on the table. When compared to the MA treatments, only the PO proportion is significantly smaller than the MA proportion (chi-squared test statistic of 7.68). In both cases, these proportions are indistinguishable from the MA2 proportion of bidders failing to make
profitable bids. If we calculate the utility of the los
ing bidders' best offer as a percentage of the maximal
utility offer available for that bidder, we find that, on
average, bidders offered 92% and 90% of the maxi mal utility offer for the PO and P02 auctions, respec
tively. If we compare the percentages of the maximum available utility offered by losing bids in the PO auc tions with those in the MA auctions, we find that, at the 5% level, only the P02 and MA2 auctions are not significantly different. Therefore, with experience, the percentage of the maximum available buyer util
ity offered by losing bids is consistent between the two auction types. The results of the rank-sum tests for these comparisons are provided in Table 3.
Conclusion 6. At least with experience, individ ual bidding behavior is similar in the MA and PO auctions.
Table 3 Wilcoxon-Mann-Whitney Rank-Sum Test Results
Comparing Proportions of Maximum Available Buyer Utility Offered by Losing Bids
Mann-Whitney p-value Test (HA) test statistic (w) (p)
PO^MA 1,075.5 0.0000 PO^MA2 684.0 0.0049 P02^MA 214.0 0.0144
P02^MA2 104.0 0.1120
Note. The left-most column represents the alternative hypothesis of the statistical test.
While individual bidding behavior is not exactly as
predicted in theory, it is relatively consistent across the two auction mechanisms. Individual behavior also
gets closer to the predicted behavior with experience in the MA auction.14 This suggests that the added
complexity of the auction environment does not lead to systematically different behavior. Variations from the theory are expected in both cases, and highlights the need to study auction design from experimental
perspective in addition to a purely theoretical devel
opment. The fact that bidders in both cases seem to behave similarly helps reconcile the superior aggre gate performance of the MA auction (compared to the PO auction) despite obvious deviations from the the oretical benchmark.
6. Conclusions and Future Work The results presented here are promising for the
application of MA auctions for procurement when several nonprice components matter. Providing bid ders with only marginal information regarding the
buyer's utility, we are able to construct a simple auc tion mechanism that clearly betters, experimentally, the PO auction format where quality and lead time are set in advance. The MA auction benefits both the
buyer and the sellers. It is only when the expected dif ferences are small that we do not observe significant improvements from the experimental MA auction.
The experimental results, by showing where real
ity differs from theory, provide potentially important insights into the MA auction design problem. Bidder
behavior was not perfect. It took experienced sub
jects to get an MA auction that closely matched, on an aggregate level, the dominant strategy predictions. Even then, individual bidding behavior was not par ticularly close to the theoretical predictions. The level of bidder losses was also not predicted. Despite these differences, the MA auction still consistently outper forms the PO auction in the lab. However, future
designs for more complex or realistic settings might
14 It is interesting to note that the same trend does not appear to be true in the PO auction.
-
Chen-Ritzo et al.: A Multiattribute Reverse Auction Mechanism with Restricted Information Feedback 1762 Management Science 51(12), pp. 1753-1762, ?2005 INFORMS
consider these deviations in an attempt to design a better MA auction.
This study has focused on the behavior of bidders in the multiattribute setting. When multiple attributes
matter, the decisions of the buyer should also be considered. An issue facing the real-world auction
designer is that the buyer may not know his utility function. Instead, he must learn about what he wants
by observing the different options available. Also, in
many supply chain auctions there are a great num ber of interdependent objects being procured simulta neously. For example, while a buyer may care about
quality and lead time, he may also care about which of two different products arrives first. These concerns
greatly complicate the multiattribute design problem and require the incorporation of combinatorial auc tion methods as discussed by Parkes and Kalagnanam (2002) and Kwasnica et al. (2005).
An electronic companion to this paper is available on the Management Science website (http://mansci. pubs.informs.org/ecompanion.html).
Acknowledgments The authors gratefully acknowledge support from the Cen ter for Supply Research (CSCR), the Institute for the Study of Business Markets (ISBM), and the Laboratory for Eco nomics Management and Auctions (LEMA) at the Smeal College of Business at The Pennsylvania State University They also thank an anonymous associate editor and two
anonymous referees. This article has benefitted from their
thoughtful input.
References
Armstrong, M. 2000. Optimal multi-object auctions. Rev. Econom. Stud. 67(3) 455-481.
Asker, J., E. Cantillon. 2004. Properties of scoring auctions. Unpub lished manuscript, Harvard University, Cambridge, MA.
Banks, J. S., J. O. Ledyard, D. P. Porter. 1989. Allocating uncer tain and unresponsive resources: An experimental approach.
RAND J. Econom. 20(1) 1-25. Beil, D. R., L. R. Wein. 2003. An inverse-optimization-based auction
mechanism to support a multiattribute RFQ process. Manage ment Sei 49(11) 1529-1545.
Bichler, M. 2000. An experimental analysis of multi-attribute auc tions. Decision Support Systems 29 249-268.
Bichler, M., J. Kalagnanam. 2005. Configurable offers and win ner determination in multi-attribute auctions. Eur. ]. Oper. Res. 160(2) 380-394.
Branco, F. 1997. The design of multidimensional auctions. RAND J. Econom. 28(1) 63-81.
Brewer, P. J., C. R. Plott. 1996. A binary conflict ascending price (BICAP) mechanism for the decentralized allocation of the right to use railroad tracks. Internat. J. Indust. Organ. 14(6) 857-886.
Che, Y.-K. 1993. Design competition through multidimensional auc tions. RAND J. Econom. 24(4) 668-680.
Chen-Ritzo, C.-H., T. P. Harrison, A. M. Kwasnica, D. J. Thomas. 2005. Better, faster, cheaper: A multi-attribute supply chain auction mechanism. Technical report, Smeal College of Busi ness Center for Supply Chain Research, The Pennsylvania State
University, University Park, PA.
Coppinger, V., V. L. Smith, J. Titus. 1980. Incentives and behavior in English, Dutch and sealed-bid auctions. Econom. Inquiry 43 1-22.
Cramton, P. 1998. Ascending auctions. Eur. Econom. Rev. 42 745-756.
Fischbacher, U. 1999. Z-Tree: A toolbox for readymade economic
experiments. Working paper no. 21, University of Zurich, Zurich, Switzerland.
Hohner, G., J. Rich, E. Ng, A. Davenport, J. Kalagnanam, H. S. Lee, C. An. 2003. Combinatorial and quantity-discount procurement auctions benefit Mars, Incorporated and its suppliers. Interfaces 33(1) 23-35.
Kagel, J. H., A. E. Roth, eds. 1995. The Handbook of Experimental Economics. Princeton University Press, Princeton, NJ.
Kagel, J. H., R. M. Harstad, D. Levin. 1987. Information impact and allocation rules in auctions with affiliated private values:
A laboratory study. Econometrica 55 1275-1304.
Koppius, O. R, E. van Heck. 2003. Information architecture and electronic market performance in multidimensional auctions.
Working paper, Erasmus University, Rotterdam, The Nether lands.
Kwasnica, A., D. Thomas. 2002. Case study: General electric trans
portation systems. W Grenoble, D. Thomas, eds. eBusiness and
Supply Chain Processes. The Supply Chain Council, Pittsburgh, PA, 40-44.
Kwasnica, A. M., J. O. Ledyard, D. Porter, C. DeMartini. 2005. A new and improved design for multi-object iterative auctions.
Management Sei. 51(3) 419-434.
Myerson, R. B. 1981. Optimal auction design. Math. Oper. Res. 6 58-73.
Parkes, D. C, J. Kalagnanam. 2002. Iterative multiattribute Vickrey auctions. Management Sei. 51(3) 435-451.
Roth, A., A. Ockenfels. 2002. Last-minute bidding and the rules for ending second-price auctions: Evidence from eBay and
Amazon auctions on the Internet. Amer. Econom. Rev. 92(4) 1093-1103.
Rothkopf, M. H., T. J. Teisberg, E. P. Kahn. 1990. Why are Vickrey auctions rare? /. Political Econom. 98 94-109.
Siegel, S., N. J. Castellan. 1988. Nonparametric Statistic for the Behav ioral Sciences. McGraw-Hill, New York.
Strecker, S. 2003. Preference revelation in multi-attribute reverse
english auctions: A laboratory study. Internat. Conf Inform. Sys tems (ICIS). Association for Information Systems, Seattle, WA, 271-282.
Williams, S. 1999. A characterization of efficient, Bayesian incentive
compatible mechanisms. Econom. Theory 14(1) 155-180.
Article Contentsp. 1753p. 1754p. 1755p. 1756p. 1757p. 1758p. 1759p. 1760p. 1761p. 1762
Issue Table of ContentsManagement Science, Vol. 51, No. 12 (Dec., 2005), pp. i-iv, 1733-1902Volume InformationFront MatterIn Memoriam: Perwez Shahabuddin, 1962-2005 [p. 1733-1733]Management Control for Market Transactions: The Relation between Transaction Characteristics, Incomplete Contract Design, and Subsequent Performance [pp. 1734-1752]Better, Faster, Cheaper: An Experimental Analysis of a Multiattribute Reverse Auction Mechanism with Restricted Information Feedback [pp. 1753-1762]The Effect of Asymmetric Bidder Size on an Auction's Performance: Are More Bidders Always Better? [pp. 1763-1776]Investment Decisions and Time Horizon: Risk Perception and Risk Behavior in Repeated Gambles [pp. 1777-1790]The Effects of Imprecise Probabilities and Outcomes in Evaluating Investment Options [pp. 1791-1803]The Effects of Financial Risks on Inventory Policy [pp. 1804-1815]Skewness Preference, Risk Aversion, and the Precedence Relations on Stochastic Changes [pp. 1816-1828]The Effect of Payoff Feedback and Information Pooling on Reasoning Errors: Evidence from Experimental Markets [pp. 1829-1843]Pricing and Resource Allocation in Caching Services with Multiple Levels of Quality of Service [pp. 1844-1859]Simple Models for Multiattribute Choice with Many Alternatives: When It Does and Does Not Pay to Face Trade-Offs with Binary Attributes [pp. 1860-1872]Tailored Supply Chain Decision Making under Price-Sensitive Stochastic Demand and Delivery Uncertainty [pp. 1873-1891]Back Matter