negotiating socially optimal allocations of resources

Negotiating Socially Optimal Allocations of Resources

U. Endriss, N. Maudet, F. Sadri, and F. Toni

Presented by: Marcus Shea

Introduction

• Consider a society of independent agents • Agents have an initial allocation of

indivisible resources• Agents can make deals with one another

in order to increase their utility

What class of deals will encourage our system to eventually reach a socially

optimal state?

Introduction

• We will examine different classes of deals– Identify necessary and sufficient classes that

will allow our society to converge to an optimal allocation

Introduction

• We will examine different classes of deals– Identify necessary and sufficient classes that

will allow our society to converge to an optimal allocation

• Examples– 1-deals without side payments– Multilateral deals with side payments

Introduction

• We will consider at different measures of social welfare– Changes definition of an ‘optimal’ allocation

Introduction

• We will consider at different measures of social welfare– Changes definition of an ‘optimal’ allocation

• Examples– Measure social welfare based on average

utility of a system– Measure social welfare based on lowest utility

of a system

Introduction

• Distributed approach to multiagent resource allocation– Local negotiation

Introduction

• Distributed approach to multiagent resource allocation– Local negotiation

• Compare to the centralized approach– Single entity decides on final allocation based

on agents preferences over all allocations– Combinatorial auctions– May be difficult to find an ‘auctioneer’

Outline

• Preliminaries• Rational Negotiation with Side Payments• Rational Negotiation without Side

Payments• Egalitarian Agent Societies• Conclusions

Preliminaries

Negotiation Framework

• Finite set of agents A • Finite set of resources R• Each agent i in A has a utility function ui that

maps every set of resources to a real number

Allocation of Resources

An allocation of resources is a function A from A to subsets of R such that A(i)∩A(j) = for i ≠ j

• An allocation of resources is just a partition of resources amongst the agents

Deals

A deal is a pair δ = (A,A’) where A and A’ are distinct allocations of resources– ‘old’ allocation and ‘new’ allocation

The set of agents involved in a deal δ = (A,A’) is given by Aδ = { i in A : A(i) ≠ A’(i) }- everyone whose set of resources has changed

The composition of two deals δ1 = (A,A’) and δ2 = (A’,A’’) is δ1◦δ2 = (A,A’’) - two deals are processed simultaneously

Independently Decomposable

A deal δ is independently decomposable if there exist deals δ1 and δ2 such that δ= δ1◦δ2 and Aδ1∩Aδ2 =

• δ is made up of two subdeals concerning disjoint sets of agents

δ =

Independently Decomposable

A deal δ is independently decomposable if there exist deals δ1 and δ2 such that δ= δ1◦δ2 and Aδ1∩Aδ2 =

• δ is made up of two subdeals concerning disjoint sets of agents

δ =

δ1δ2

δ = δ1◦δ2

Utility Functions

• We may restrict our attention to utility functions ui with particular properties:– Monotonic: for all R1,R2 R– Additive: for all R R

– 0-1 Function: Additive and for all r in R – Dichotomous: for all R R

1 2 i 1 i 2R R u (R ) u (R )

i ir R

u (R) u ({r})

iu ({r}) {0,1}iu (R) {0,1}

Utility Functions

• We may restrict our attention to utility functions ui with particular properties:– Monotonic: for all R1,R2 R– Additive: for all R R

– 0-1 Function: Additive and for all r in R – Dichotomous: for all R R

• An agent’s utility of an allocation is just the utility of his set of resources ui(A) = ui(A(i))

1 2 i 1 i 2R R u (R ) u (R )

i ir R

u (R) u ({r})

iu ({r}) {0,1}iu (R) {0,1}

Rational Negotiation with Side Payments


• We consider the scenario where agents can exchange money as well as resources

• We define a payment function as a function p from agents to real numbers that, when summed over agents, equals zero:

i

p(i) 0A


Our goal is to maximize utilitarian social welfare

• Utilitarian social welfare is just the sum of all agents utility– Maximizing is equivalent to maximizing average utility– Useful in any market where agents act individually

u ii

sw (A) u (A)A

Individually Rational

• We assume our agents are rational

• We say a deal is individually rational if there exists a payment function so that every involved agent’s increase in utility is strictly greater than their payment

• Formally: deal δ = (A,A’) is individually rational if there exists a payment function p such that ui(A’) – ui(A) > p(i) for all agents i, except possibly p(i) = 0 for agents with A(i) = A’(i)

1-deals

A 1-deal is a deal involving reallocation of exactly one resource

• Question: If (rational) agents are permitted to perform 1-deals only, will we eventually reach an optimal allocation?

1-deals

• Consider a system with two agents and two resources, r1 and r2

• We specify the utility functions:

• Initial allocation A: Agent 1 has both resources

u1({}) = 0 u2({}) = 0

u1({r1}) = 2 u2({r1}) = 3

u1({r2}) = 3 u2({r2}) = 3

u1({r1,r2}) = 7 u2({r1,r2}) = 8

1-deals

• Consider a system with two agents and two resources, r1 and r2

• We specify the utility functions:

• Initial allocation A: Agent 1 has both resources– swu(A) = 7, optimal allocation has value 8

– 1-deals are not sufficient to get to an optimal allocation

u1({}) = 0 u2({}) = 0

u1({r1}) = 2 u2({r1}) = 3

u1({r2}) = 3 u2({r2}) = 3

u1({r1,r2}) = 7 u2({r1,r2}) = 8

First Result

• We are going to move toward showing that if we allow our agents to perform arbitrary individually rational deals, then we will reach an optimal allocation through negotiation

Lemma 1

Lemma 1: A deal δ = (A,A’) is individually rational iff swu(A) < swu(A’)

• Intuition: If an entire society gets a strict

increase in utility, then those profiting can payoff those who are losing so that everyone shares the gain

Thm 1: Maximal Utilitarian Social Welfare

Theorem 1: Any sequence of individually rational deals will eventually result in an allocation A that maximizes swu(A)



Proof:



Proof:• Termination Argument

– A and R finite means that there are only finitely many allocations

– Lemma 1 gives that any individually rational deal strictly increases social welfare



Proof:



Proof:• Suppose terminal allocation A is such that

swu(A) < swu(A’) for some A’



Proof:• Suppose terminal allocation A is such that

swu(A) < swu(A’) for some A’ ≠ A• Then deal δ = (A,A’) increases social

welfare, and thus is individually rational by Lemma 1, contradicting termination


• Implications of Theorem 1– Not really surprising

• Class of individually rational deals allows for any number of resources to be moved between any number of agents




– Difficulty in actually finding an individually rational deal





– We will not get stuck in a local optimum, any sequence will bring us to optimum allocation





– We will not get stuck in a local optimal, any sequence will bring us to optimum allocation

– This sequence could, however, be very long

Do we need the entire class of individually rational deals to guarantee that negotiation

will eventually reach a socially optimal allocation?

Thm 2: Necessary Deals w/ Side Payments

Theorem 2: Fix A, R. For every deal δ that is not independently decomposable, there exist utility functions and an initial allocation so that any sequence of individually rational deals leading to an optimal allocation must include δ.



• This remains true if we restrict utility functions to be monotonic, or dichotomous



• This remains true if we restrict utility functions to be monotonic, or dichotomous

Proof: Carefully define utility functions and initial allocation so that δ is the only improving deal


• Implications of Theorem 2– Any negotiation protocol that puts restrictions

on the structural complexity of deals will fail to guarantee optimal outcomes if the class of utility functions is unrestricted, monotone, or dichotomous




• What can we do?




• What can we do?– Restrict utility functions– Change notion of social welfare

Additive Scenario

• Consider the scenario where utility functions are additive (no synergy effects)

• Will we be able to reach an optimal allocation without needing such a broad class of deals?

Thm 3: Additive Scenario

Theorem 3: In additive scenarios, any sequence of individually rational 1-deals will eventually result in an allocation with maximal utilitarian social welfare



Proof:



Proof:• We get termination since we are looking

at individually rational deals


Proof:

R

For any allocation , define the function R A to

simply tell us the agent that holds resource in allocation

That is,

We can write since utility functions arA

A

A

u f (r )r

A f :

r A

f (r) i r A(i)

sw (A) u ({r})

e additive


Proof:

R



That is,


A

A

u f (r )r

A f :

r A

f (r) i r A(i)

sw (A) u ({r})

e additive

Suppose negotiation terminates with allocation but another

allocation with higher social welfare,

By the function above, some resource R must be generating

higher utility un

u u

u

A

A ' sw (A ') sw (A)

sw r

der that new allocation than under allocation

That is, A ' Af (r ) f (r )

A ' A

u ({r}) u ({r})


Proof:

R



That is,


A

A

u f (r )r

A f :

r A

f (r) i r A(i)

sw (A) u ({r})

e additive

Suppose negotiation terminates with allocation but another

allocation with higher social welfare,

By the function above, some resource R must be generating

higher utility un

u u

u

A

A ' sw (A ') sw (A)

sw r

der that new allocation than under allocation

That is, A ' Af (r ) f (r )

A ' A

u ({r}) u ({r})

Then the 1-deal passing from agent to increases social welfare

By Lemma 1, is an individually rational 1-deal, contradicting terminationA A 'r f (r) f (r)


• Are 1-deals necessary to achieve an optimal allocation in the additive scenario?


• Are 1-deals necessary to achieve an optimal allocation in the additive scenario?– Paper does not address this question


• Are 1-deals necessary to achieve an optimal allocation in the additive scenario?– Paper does not address this question– Easy to see that they are necessary:

• Let δ be a 1-deal that moves resource r1 from agent i to agent j

• Give all resources to agent j, except r1 to agent i• Set uk({r}) = 0 for every resource r, every agent k≠j• Set uj({r}) = 1 for every resource r• Only individually rational deal is 1-deal δ

Class of DealsSidePayments

Utility Functions

Measure of Social Welfare

Nature of Optimality

Necessary / Sufficient

Individually Rational Deals Yes

UnrestrictedMonotonicDichotomous Utilitarian

Global Maximum

Sufficient[1] & Necessary[2]

Individually Rational 1-deals Yes Additive Utilitarian

Global Maximum

Sufficient[3] & Necessary

Cooperatively Rational Deals No


Pareto Optimal


Cooperatively Rational 1-deals No 0-1 Functions Utilitarian

Global Maximum

Sufficient[6]& Necessary

Equitable Deals NoUnrestrictedDichotomous Egalitarian

Global Maximum


Simple Pareto-Pigou-Dalton Deals No 0-1 Functions Mixed

Lorenz Optimal Sufficient[9]

Summary of Results

Rational Negotiation without Side Payments

Rational Negotiation w/o Side Payments

• Now we consider the scenario where there are no side payments made


• Now we consider the scenario where there are no side payments made

• The class of individually rational deals no longer allows us to achieve optimal social welfare:– Agent 1 has sole resource r

u1({}) = 0 u2({}) = 0

u1({r}) = 1 u2({r}) = 2


• Maximizing social welfare is no longer possible in general

• We will instead see if a Pareto optimal outcome is possible, and what types of deals are sufficient to guarantee this outcome

Pareto Optimal

• A Pareto optimal allocation is one in which there is no other allocation with higher social welfare that would be no worse for any of the agents in the system

Formally: Allocation A is Pareto optimal if there is no allocation A’ such that swu(A) < swu(A’) and ui(A) ≤ ui(A’) for all agents i

Pareto Optimal

• Recall our previous example – Agent 1 has sole resource r

– This is Pareto optimal since agent 1 is worse off by giving resource r to agent 2, even though it would increase social welfare

u1({}) = 0 u2({}) = 0

u1({r}) = 1 u2({r}) = 2

Cooperative Rationality

We say a deal is cooperatively rational if no agent’s utility decreases, but at least one agent’s utility strictly increases

Formally: We say a deal δ = (A,A’) is cooperatively rational if ui(A) ≤ ui(A’) for all agents i and there is an agent j such that uj(A) < uj(A’)

• We examine the class of cooperatively rational deals for the scenario without side payments

Thm 4: Pareto Optimal Outcomes

Theorem 4: Any sequence of cooperatively rational deals will eventually result in a Pareto optimal allocation of resources

• Very similar proof to Theorem 1

Thm 5: Necessary deals w/o side payments

Theorem 5: Fix A, R. Then for every deal δ that is not independently decomposable, there exist utility functions and an initial allocation such that any sequence of cooperatively rational deals leading to a Pareto optimal allocation would have to include δ


Theorem 5: Fix A, R. Then for every deal δ that is not independently decomposable, there exist utility functions and an initial allocation such that any sequence of cooperatively rational deals leading to a Pareto optimal allocation would have to include δ

• Still holds if utility functions are restricted to be monotonic or dichotomous


• Analogously to Theorem 3, we can restrict our utility functions to get a positive result about converging to an optimal solution under the class of cooperatively rational 1-deals

Thm 6: 0-1 Scenarios

Theorem 6: If utility functions are 0-1 functions (additive and ui({r}) = 0 or 1), any sequence of cooperatively rational 1-deals will eventually result in an allocation with maximal utilitarian social welfare



• Note that we actually get optimal social welfare in this case, not just Pareto optimal!



• Note that we actually get optimal social welfare in this case, not just Pareto optimal!

• Proof is simple– If A is not optimal, must have a agents i and j and

resource r where r is in A(i), ui({r}) = 0 and uj({r}) = 1– That 1-deal is cooperatively rational


Utility Functions






Global Maximum



Global Maximum




Pareto Optimal



Global Maximum

Sufficient[6]& Necessary


Global Maximum




Summary of Results

Egalitarian Agent Societies

Egalitarian Social Welfare

Consider a new measure of social welfare called egalitarian social welfare

Ae isw (A) min{u (A) | i }



• Measures the utility of the ‘weakest/poorest’ member of the society




• Measures the utility of the ‘weakest/poorest’ member of the society

• Makes sense when the society is working together or trying to be fair with one another– Recall: Earth Observation Satellite Access


Equitable Deals

A deal δ = (A,A’) is equitable if

min{ ui(A) | i in Aδ } < min{ ui(A’) | i in Aδ}

Equitable Deals



• Lowest utility of all agents involved in a deal increases

Equitable Deals



• Lowest utility of all agents involved in a deal increases• Note: we do not need the weakest member of society to

improve– Would not be a local condition

Equitable Deals


min{ ui(A) | i in Aδ } < min{ ui(A’) | i in Aδ }

• Lowest utility of all agents involved in a deal increases• Note: we do not need the weakest member of society to

improve– Would not be a local condition

Lemma 2: If A and A’ are allocations with swe(A) < swe(A’), then δ = (A,A’) is equitable

Thm 7: Maximal Egalitarian Social Welfare

Theorem 7: Any sequence of equitable deals will eventually result in an allocation of resources with maximal egalitarian social welfare

Thm 7: Maximal Egalitarian Social Welfare

Theorem 7: Any sequence of equitable deals will eventually result in an allocation of resources with maximal egalitarian social welfare

• Only difficulty of proof is showing termination, the rest comes from the definition of equitable

Thm 8: Necessary Deals in Egalitarian Systems

Theorem 8: Fix A, R. Then for every deal δ that is not independently decomposable, there exist utility functions and an initial allocation such that any sequence of equitable deals leading to an allocation with maximal egalitarian social welfare would have to include δ.

Thm 8: Necessary Deals in Egalitarian Systems

Theorem 8: Fix A, R. Then for every deal δ that is not independently decomposable, there exist utility functions and an initial allocation such that any sequence of equitable deals leading to an allocation with maximal egalitarian social welfare would have to include δ.

• Still holds if utility functions are restricted to be dichotomous


Utility Functions






Global Maximum



Global Maximum




Pareto Optimal



Global Maximum



Global Maximum




Summary of Results

Conclusions

• We studied an abstract negotiation framework where members of an agent society arrange multilateral deals to exchange bundles of indivisible resources

Conclusions

• We studied an abstract negotiation framework where members of an agent society arrange multilateral deals to exchange bundles of indivisible resources

• We analyzed how the resulting changes in resource distribution affect society with respect to different social welfare orderings

Conclusions

• We see that convergence to an optimal allocation depends on:

Conclusions

• We see that convergence to an optimal allocation depends on:– the class of allowable deals

Conclusions

• We see that convergence to an optimal allocation depends on:– the class of allowable deals– the notion of optimality being considered

Conclusions

• We see that convergence to an optimal allocation depends on:– the class of allowable deals– the notion of optimality being considered– the restrictions on utility functions

Conclusions

• We see that convergence to an optimal allocation depends on:– the class of allowable deals– the notion of optimality being considered– the restrictions on utility functions– the availability of side payments

Conclusions

• We see that convergence to an optimal allocation depends on:– the class of allowable deals– the notion of optimality being considered– the restrictions on utility functions– the availability of side payments

• Natural question: Complexity results– How fast do we converge to the optimal

allocation? [Endriss and Maudet (2005)]

Conclusions

• Authors are looking at welfare engineering– Application-driven choice of a social welfare

ordering– Design of agent behaviour profiles and

negotiation mechanisms that permit socially optimal outcomes

Questions?

Lorenz Domination

• Let A, A’ be allocations for a society with n agents. Then A is Lorenz dominated by A’ if

and furthermore, that inequality is strict for at least one k.

• k = 1 gives egalitarian social welfare• k = n gives utilitarian social welfare

k k

i ii 1 i 1

u (A) u (A ') k 1,...,n

############################

Pigou-Dalton Transfer

• A deal δ = (A,A’) is called a Pigou-Dalton transfer if it satisfies:– 2 agents involved– Mean-preserving: ui(A) + uj(A) = ui(A’) + uj(A’)– Reduces inequality:

|ui(A’) – uj(A’)| < |ui(A) – uj(A)|• A simple Pareto-Pigou-Dalton deal is a 1-

deal that is either cooperatively rational, or a Pigou-Dalton transfer


Utility Functions






Global Maximum



Global Maximum




Pareto Optimal



Global Maximum



Global Maximum




Summary of Results

negotiating socially optimal allocations of resources

Documents