example: standard (first-price) second-price sealed-bid auction...

1

1

Game Theory/Mechanism Design/Auctions

in Multi-agent Systems

Makoto Yokoo

Kyushu University, Japan

[email protected]

http://agent.inf.kyushu-u.ac.jp/~yokoo/

2

Outline• Examples

• Motivation (why game theory?)

• Basic Terms

• Single-item Auctions

• Combinatorial Auctions, VCG, & False-name Bids

• Automated Mechanism Design

3

Example: Standard (First-price) Sealed-bid Auction

Mechanism: Each bidder (agent) privately bids the price it is willing to pay, and the highest bidder wins; pays the value of its bid.

$8000 $6000$7000

Demerit: Spying other agents’ bids is profitable.

4

Second-price Sealed-bid Auction (Vickrey Auction)

Mechanism : The highest bidder wins, but pays the value of the second highest bid.

$8000 $6000$7000

Pay the second highest bid ($7000)

Characteristic:

–Bidding its true evaluation value (the maximal value where it does not want to pay any more) is the optimal strategy (incentive compatibility).

–Spying other agents’ bids is meaningless.

5

Spying is Useless Because... Assume its own evaluation value is $8000, and found

that others’ highest bid is:

(I) less than $8000: its payment is the same to the case when it submits the true evaluation value (without spying).

(II) more than $8000: the agent cannot obtain positive utility anyway.

Its profit cannot be more than the case when it truthfully submits the true evaluation value.

$8000 $6000$7000

Pay the second highest bid ($7000)

6

Quiz: The larger also serves for the smaller?

• If the highest bidder wins, and pays the value equals to the third highest bid, is honesty still the best policy?

$150

$50

$100

2

7

Answer

• Honesty is no longer the best policy.

– For the first and second bidder, if he wins, he is going to pay only $50.

– Can increase his bid to infinity to beat the opponent.

• If two identical units are sold (i.e., both are winners), then honesty is the best policy.

8

Outline• Examples


• Basic Terms




9

Why Game-theory/Economics?

• Game-theory/Economics provide analytical tools for the design/analysis of multi-agent systems (MAS).

– Game-theory/economics are specially useful when the MAS

• are not centrally designed.

• do not have a notion of global utility.

• will not necessarily act “benevolently”.

10

Why Game-theory/Economics (contd.)?

• Underlying assumptions:

–Each player (agent) has consistent preference/utility.

–Each player is rational, i.e., tries to maximize his/her utility.

–Each player chooses a strategy to play.

These assumptions only approximate human behavior, but are fully applicable to MAS.

11


• Increasing intersectional topics:

– Internet auctions, electronic commerce

– resource allocation for automated agents

From the theory for describing human behavior to the theory for designing systems!

John von Neumann

12


• MAS research community is divided into several different groups (like clubs).

– e.g., logic, search/constraint satisfaction, game-theory

• Each club has its own tradition.

• The members of each club speak their own language.

• You need to join a club to publish your paper.

• To join a club, you need to speak their language and understand their tradition.

• Game-theory/mechanism design is one of mainstream clubs in MAS.

3

13

Outline• Examples


• Basic Terms




14

Characteristics of Player

risk neutral/averse

• risk neutral： only cares the expected utility

– e.g., indifferent between two lotteries: 1) he/she obtains 0 for the head and $100 for the tail,2) he/she obtains $50 for sure.

• risk averse： prefers that is more certain （even with less expected utility）

– e.g., prefers getting $45 for sure to 1.

15

Assumption for Simplicity

Quasi-linear utility: an agent’s utility is defined as the difference betweenits evaluation value of the allocated good

and its payment.

• If an agent wins a good whose evaluation value is $100 by paying $90, its utility is $100-$90=$10.

• If it does not win, its utility is $0.

• If it does win, paying $100, its utility is 0.

16

The St. Petersburg Paradox

Assume the following lottery...

• Flip a coin, if it comes up tail, you get $2.

• If head, flip a coin again, if it comes up tail, you get $4.

• If head, flip a coin again, ....

• If it comes up tail the first time at n-th trial, you get $2n.

How much are you willing to pay to participate this lottery?

17

Strategy & EquilibriumStrategy: the way for choosing actions, i.e., how to

decide the bidding price

Dominant Strategy: the best strategy regardless of the actions of other agents

– Paper-rock-scissors: no dominant strategy

– If only paper and rock are allowed, paper is the dominant strategy

Dominant Strategy equilibrium: assume each agent has a dominant strategy. Then, the combination of dominant strategies is called dominant strategy equilibrium.

Nash equilibrium: a weaker concept than a dominant-strategy equilibrium. A combination of strategy is a Nash equilibrium if each strategy is a best response to other strategies.

18

Desirable Properties of a Group Decision Making Mechanism

• For each agent, there exists a dominant strategy.

• The mechanism is robust against various frauds (e.g., spying).

• A Pareto efficient outcome can be achieved.

4

19

Pareto EfficiencyDefinition:

A state is said Pareto efficient, iff there exists no other state that is

– better for one agent, and

– no worse for all the rest

Movie 2 2 2

Shopping 2 2 5

Zoo 2 3 1

Home 1 1 1×

×

20

Social Surplus• Assuming agents’ utilities are quasi-linear,

if a state is Pareto efficient, the social surplus (sum of all agents’ utilities) must be maximized.

Movie 2 2 2

Shopping 2 2 5

Zoo 2 3 1

Home 1 1 1

×

×

→3 →4

×

6963

21

Pareto Efficiency in Auctions

• The social surplus must be maximized; the good must be allocated to the agent who has the highest evaluation value.

• The agent whose evaluation value is $8000 wins and pays $7000.

• Utility of this agent: $8000 - $7000=$1000

• Utility of the seller: $7000

• Social Surplus: $8000

$8000 $6000$7000

22

Private/Common/Correlated Values

Private Value: each agent knows its value with certainty, which is independent from other agents’ evaluation values (e.g., antiques which are not resold).

Common value： the evaluation values for all agents are the same, but agents do not know the exact value and have different estimated values (e.g., US Treasury bills, mining right of oil fields).

Correlated Value: Something between above two extremes.

23

Mechanism Design• Designing a mechanism is

determining the rules of a game.

• The designer cannot control the actions of each agent.

–No way to force an agent to be honest or to refrain from doing frauds.

24

Mechanism DesignHow can a designer achieve a certain desirable

property (e.g., Pareto efficiency)?

• Design rules so that:

– For each agent, there exists a dominant strategy.

– In the dominant strategy equilibrium, the desirable property is achieved.

5

25

Outline• Examples


• Basic Terms




26

First-price Sealed-bid

Mechanism：Each agent submits its bid without knowing other agents’ bids. The agent with the highest bid wins and pays its own price.

Dominant strategy： does not exist in general.

27

Example: First-price

• Assume you attend an auction on behalf of your uncle.

• Your uncle specifies the maximal price you can bid for the good ($100).

• The generous uncle will give you the difference if you can buy it less than $100.

• If you failed to buy a good, you receive nothing for the good.

28

Example: First-price

• Assume there is only one opponent.

• Your opponent is also a proxy.

• You don’t know how much your opponent will bid, but you know his maximal price is uniformly distributed among [0, 200].

29

Strategy for Bidding

• Clearly, bidding more than $100 is meaningless (you must pay the difference).

• If you bid $90 and win, your profit is $10.

• If you bid $1 and (by any chance) win, your profit is $99.

• How much should you bid?

30

• We cannot tell the expected utility unless knowing the strategy of the opponents.

• Let us assume the bidding price of the opponents is uniformly distributed between [0, 100]

• If you bid low, the probability of winning is low, but the utility when wins is large.⇒high-risk, high-return

• If you bid high, the probability of winning is high, but the utility when wins is low.⇒low-risk, low-return

• The best point is in between, bidding the half (50), where the expected utility is 25.

Answer

１００５０

２５

bidding price

utility

6

31

• If the opponent uses the same strategy, his bid is uniformly distributed between [0, 100], since his maximal price is uniformly distributed between [0, 200].

• This pair of strategies is (Bayesian) Nash equilibrium, i.e., the best response with each other.

• Even if you are not a proxy, i.e., the maximal price is your own evaluation value, you should use the same strategy.

Answer (contd.)

32

Dutch (descending)

Mechanism: The seller announces a very high price, then continuously lowers the price until some agent says “stop”, then the agent wins the good and pays the current price.

Dominant strategy： does not exist in general.

33

Dutch (descending)

• Strategically equivalent to the first-price sealed-bid auction

– there is one-to-one mapping between the strategy sets in two

auctions.

• Example:

– Dutch flower market

– Ontario tobacco auction

– bargain sale

$8000 $6000$7000

89008400790074006900

STOP!

34

English (open cry)Mechanism：Each agent is free to revise

its bid upwards. When nobody wishes to revise its bid further, the highest bidder wins the good and pays its own price.

(Almost) Dominant strategy (in private value): keep bidding some small amount more than the previous highest bid until the price reaches its evaluation value, then quit.

35

English (open cry)

• In everybody uses the minimum increment strategy, the agent with the highest evaluation value wins and pays the second highest evaluation value +ε.

• The obtained allocation is Pareto efficient.

$8000 $6000$7000

6000

7000

8000

7000

6000

36

Vickrey (Second-price Sealed-bid)Mechanism：Each agent submits its bid without

knowing other agents’ bids. The agent with the highest bid wins and pays the value of the second highest bid.

Dominant strategy (in Private value)： Bidding its true evaluation value is a dominant strategy

• The obtained allocation in a dominant strategy equilibrium is Pareto efficient.

• The obtained result is identical to English in the dominant-strategy equilibrium.

7

37

Characteristics of mechanisms (in Private Value Auctions)

• Dutch＝First-price Sealed-bid

• English＝Vickrey

• Under several assumptions, the expected revenue of the seller is the same in all 4 auction mechanisms (revenue equivalence theorem, Vickrey 1961）．

– If there exists a Bayesian Nash equilibrium, the expected revenue would be the same at the equilibrium．

38

Difficulties for Using Vickrey Auction

• Hard to understand!

• Do not know/aware of the true evaluation value (even in private value auctions).

• Cannot trust the seller.

• Do not want to reveal the private/sensitive information.

Is mechanism design/auction theory useless?

39

Sponsored Search40

Sponsored Search

• An advertiser bids for a search keyword.

• When a keyword is searched by a user, links to advertisers are shown in the decreasing order of bids.

• It enables an advertisement targeted for a particular type of users.

• An advertiser pays only when a user actually clicks the link （pay-per-click）.

Question: how to set the payment?

41

Payment Scheme• In early systems, an advertiser pays the amount of her

bid (first-price).

– Setting an appropriate amount of a bid is difficult.

– An advertiser can send a dummy search request and adjust her bid.

• The payment scheme is changed so that an advertiser who obtains k-th slot pays the amount of k+1-st bid.

Lessons Learned:

• Second-price/Vickrey auction, was never widely used, now became the most frequently used auction mechanism in the world.

• For a mechanism, which is frequently used on the Internet by users including automated agents, having theoretical robustness is crucial.

42

Outline• Examples


• Basic Terms




8

43

Combinatorial Auction

• Multiple different goods with correlated values are auctioned simultaneously.

– Complementary: PC and memory

– Substitutable: Dell or Lenovo

• By allowing bids on any combinations of goods, the obtained social surplus/revenue of the seller can increase.

• e.g., FCC spectrum right auctions

44

Research Issues in Combinatorial Auctions

• Finding the best combination of bids is a complicated combinatorial optimization problem – winner determination problem, one instance

of a set packing problem

– NP-complete

– Various search techniques are introduced

• How to describe the preference of an agent is also a research issue --- 2m

subsets for m goods

45

Vickrey-Clarke-Groves Mechanism (VCG) aka Generalized Vickrey

• Each agent declares its evaluation values for subsets of goods.

• The goods are allocated so that the social surplus is maximized.

• The payment of agent 1 is equal to the decrease of the social surplus except agent 1, caused by the participation of agent 1.

• Satisfies incentive compatibility and Pareto efficiency.

46

An Example of the VCGSetting: three agents (agent 1, 2, 3) are

bidding for two goods.

Result:•Agent 1 gets the coffee, 3 gets the cake.•Agent 1 pays $8－$5＝$3.•Agent 3 pays $8－$6＝$2.

coffee cake both

Agent 1 $6 $0 $6

Agent 2 $0 $0 $8

Agent 3 $0 $5 $5

47

1‘s utility ($3)

social surplus when 1 does not participate ($8)

1’s payment ($3)

1‘s evaluation value ($6)

social surplus except 1 when 1 does participate ($5)

Social Surplus ($11)

Incentive Compatibility of VCG• Goods are allocated so that social surplus is maximized.

• An agent can maximize its utility when the social surplus is maximized.

Limitations of VCG• The VCG can be used for mechanism design in

general. It has theoretically well-founded.

– satisfies (dominant-strategy) incentive compatibility, achieves Pareto efficient allocation in a dominant strategy equilibrium.

• However, it has several limitations.

– Computationally expensive (NP-hard), the outcome is not in the core, vulnerable against the collusion of losers, vulnerable against false-name bids.

– It cannot guarantee budget balance/non-negative in more general settings (apart from auctions).

We still need to design new mechanism for each application domain. 48

9

49

bids

False-name bids

False-name Bids(Yokoo, et al. GEB-2004)

An agent submits several bids under fictitious names.

• Detecting false-name bids is virtually impossible, since identifying each participant on the Internet is very difficult.

50

A Case where the VCG is Vulnerable

Setting: two agents (Agent 1, 2)

When telling the truth:

•Agent 1 gets both goods.

•payment: $8 －$0 ＝$8

coffee cake both

Agent 1 $6 $5 $11

Agent 2 $0 $0 $8

coffee cake both

Agent 1 $6 $0 $6

Agent 2 $0 $0 $8

Agent 3 $0 $5 $5

When agent 1 uses a false-name 3 and splits its bid:•Agent 1 gets both goods.•payment: $3+$2=$5

51

Outline• Examples


• Basic Terms




52

Mechanism Design• A mechanism M can be

considered as a function, which takes a type profile q as an input, and returns the result of social choice (e.g., an allocation o and payment p)

• We want the function to optimize some objective (e.g., social surplus, revenue), while it satisfy some constraints (e.g., incentive compatibility, individual rationality).

• Traditionally, such a function is hand-crafted.

Input q

(valuations of bidders)

Social Choice

（allocation o

and payment p）

Mechanism M

（Auction Mechanism）

Automated Mechanism Design(Conitzer & Sandholm, UAI-2002)

• Formalize the mechanism design problem as an optimization problem (mixed integer programming problem, MIP).

– Introduce (many) decision variables, each of which represents a possible combination between an input and an output.

– Determine the values of these variables, so that the objective (e.g., social surplus) is maximized under given constraints (e.g., incentive compatibility).

• Once a problem is formalized as a MIP, an off-the-shelf optimization package can solve the problem (e.g., CPLEX by ILOG/IBM)

53

Example: Auction with 2 bidders and 1 good

• Bidder 1 and 2 bid for one good.

• Possible valuation： $100 or $50

• Possible assignment for each bidder： win or lose

• Possible type profile：(q1, q2) = (100, 100), (100, 50), (50, 100) or (50, 50)

• Possible allocation:

• (o1, o2) = (win, lose), (lose, win) or (lose, lose)

• Payment （non-negative real value）

• For each input, a mechanism determines the allocation and payment:

– (q1, q2) → (o1, o2) , (p1, p2) 54

10

Variables in MIP FormalizationVariables for allocations：

Example：prob_(100, 50)_(win, lose)： represents the probability that for an input/bid (100, 50), the allocation will be (win, lose).

• If a mechanism is deterministic, its value is 0/1.

• In short, we enumerate all possible combinations of an input and an output, and assign 0/1 values.

Variables for payments：Example：p1_(100, 50): represents the payment of bidder 1 for an input/bid (100, 50).

• Its value is a non-negative real value.

Constraints in MIP Formalization（Incentive Compatibility）

• A bidder’s utility does not increase by mis-reporting.

• Expected utility for bidder 1 (whose valuation is 100), and reported types are (100, 50):

– 100*prob_(100, 50)_(win, lose) + 0*prob_(100, 50)_(lose, win) + 0*prob_(100, 50)_(lose, lose)

– p1_(100, 50)

• Expected utility when bidder 1 mis-reports her valuation as 50:

– 100*prob_(50, 50)_(win, lose) + 0*prob_(50, 50)_(lose, win) + 0*prob_(50, 50)_(lose, lose)

– p1_(50, 50)

≧

Result of Automated Mechanism Design (AMD)

• The objective (e.g., maximizing social surplus/revenue) is represented as a linear function.

• Variable values are determined so that they optimize the objective, while satisfying given constraints.

Social Surplus Revenue

(100, 100) (win, lose), (100, 0) (win, lose), (100, 0)

(100, 50) (win, lose), (50, 0) (win, lose), (100, 0)

(50, 50) (win, lose), (50, 0) (lose, lose), (0, 0)

Vickrey? Reserve price=100?

Advantages of AMD

• It is very flexible:

– We can choose any constraints: dominant-strategy incentive compatibility, Bayesian Nash IC, budget balance/non-negative, individual rationality（ex post, interim, ex ante）, ...

– We can choose any objective: social surplus, revenue, competitive ratio, ...

– We can choose the type of a mechanism: deterministic/non-deterministic, ...

• It obtains a custom-made mechanism, which is specialized to the possible range of inputs.

Can we design a mechanism automatically? No need for human mechanism designers?

58

Limitations of AMD

• Types must be discretized for combinatorial optimization （e.g., a real value between 0 and 100 → an integer between 0 and 100）.

• The problem size of MIP formalization will explode soon.

• Assume # of participants is n, # of possible types is t, then # of possible type profiles is tn.

• Assume # of goods is m, then, # of possible allocations is (n+1)m.

• # of variables for allocations becomes tn ・ (n+1)m.

• When n=4, m=3, t=9, it is more than 820 thousand.

• CPLEX cannot handle such a large-scale MIP instance.

We cannot find a good custom-made mechanism for a realistic size → Is automated mechanism design totally useless?

59

How can We utilize AMD?• We can use AMD for obtaining some hints/intuition for

designing a general mechanism.

• Select representative input values (e.g., high, middle, low)

• Run AMD for a small number of participants.

• Examine the result/table, and extract some characteristic parts (e.g., different from existing mechanisms).

• Extract a generalized rule for allocation/payment by human intuition.

• Examine the extracted rule.

• If it does not work, change input values and run another round.

Although we don’t mention about AMD in our paper, we secretly use AMD (other computer scientists also?).

60

11

Concluding Remarks

• Game theory becomes a prerequisite subject for most AI and MAS researchers.

• Designing a mechanism is designing a function, but there exist constraints between not only the current input and output, but between possible outputs of different inputs.

• If you like designing an algorithm, you would like designing a mechanism also.

62

Further Readings (books) • Textbook on MAS including game theory:

– Yoav Shoham, Kevin Leyton-Brown, MultiagentSystems: Algorithmic, Game-Theoretic, and Logical Foundations, 2008

• Textbook on Auction:

– Vijay Krishna, Auction Theory, Academic Press, 2002.

• Textbook on Combinatorial Auctions

– Combinatorial Auctions, Peter Crampton, YoavShoham, Richard Steinberg, eds., MIT Press, 2006.

• Mid-level textbooks on economics in general:

– Andreu Mas-Colell, Michael D. Whinston and Jerry R. Green, Microeconomic Theory, Oxford University Press, 1995.

example: standard (first-price) second-price sealed-bid auction...

Documents