sabita maharjan energymarket and gametheory slides · 2018. 9. 19. · cournot,bertrand&...

ENERGY MARKET AND GAME THEORY

SabitaMaharjanSenior Research ScientistSimulaMetropolitan Center for Digital Engineering (CDE, SimulaMet), Associate Professor,Department of Informatics, University of Oslo, Norway

September 7, 2018

Outline

2

Demand Response Management and Energy Market

Game Theory: Examples and Introduction

Discussions

STARTING WITH SOME EXAMPLESGame Theory

Battle of the Sexes

You and your partner are spending weekend together.

You want to play an online game

Your partner wants to go shopping.

What will you choose?

Selection of Power Sources

Oslo has two power companies Aand B to provide electricity

– A is stable, but not green

– B is green, but not stable

AB

How should a user choose A or B?

Common features in these examples

§ Competitive situations or situations of conflict

§ Multiple players

§ Decision making

§ Concerned more about choices than ‘best’ solutions

GAME THEORYBackground, Basic Concepts and Definitions

Game Theory: History

Cournot, Bertrand and Stackelberg

8

1944

John Von Neuman and Oskar Morgensten: Game theory became more widely known

1928

John Von Neumann

1950-‐1960

Important Developments: Nash equilibrium

1994

John Harsanyi, John Nash and Rienhard SeltenWere awarded Nobel Prize for Economics

Game theory as a discipline

Game theory is an interdisciplinary field encompassing economics and mathematics and one that can be deployed to solve problems for numerous applications

Strategies

Three key elements in a game

Players Payoffs

Figure source: http://www.slate.com/articles/sports/sports_nut/2013/11/the_world_chess_championship_is_an_embarrassing_anachronism_it_s_time_to.html

Game theory as a discipline

Game theory is an interdisciplinary field encompassing economics and mathematics and one that can deployed to solve problems for numerous applications

Three key elements in a game

Players: Each player can be an individual, a group or an organization

Strategies: a plan of actions by which a player has a decision rule to determine their moves for every possible situation in a game

Payoff: benefit received for a given strategy; often termed as utility

Figure source: http://www.slate.com/articles/sports/sports_nut/2013/11/the_world_chess_championship_is_an_embarrassing_anachronism_it_s_time_to.html

Key Elements in “Battle of the Sexes” – an example

or

Players

Strategies

Payoff

You and your partner

{Playing video game},{going shopping}

happiness

Two types of games

Non-‐cooperative game

A game with competition between individual players. Only self-‐enforcing (e.g. through credible threats) alliances (or competition between groups of players) are possible due to the absence of external means to enforce cooperative behavior (e.g. contract, law).

Cooperative game

A game with competition between groups of playersdue to the possibility of external enforcement of cooperative behavior (e.g. through contract)

Cooperation generally leads to higher payoffs. For example: countries cooperate on trading (reduced tariffs) leading to boost in exports

A classic non-‐cooperative game “Prisoner’s Dilemma”

You and your friend Bill are arrested and thrown into prison in separate cells.

For three days, neither you nor Bill has told nothing

………….Then……………

You are told by the police: “We have your friend Bill and he is starting to talk”

Bill Gates at the age of 19Should you confess?

Figure source: https://imgur.com/gallery/qnK6woX

How to choose strategy to minimize the number of years in prison?

Each prisoner is rational and selfish. Namely, he wants to maximize his own benefit and does not care about the other person’s benefit.

Each prisoner is put in separate room and does not know the other person’s choice.

When you and Bill can talk, you may not trust even if Bill claims to cooperate.

Prisoner’s Dilemma – payoff matrix

Prisoner’s Dilemma – payoff matrix

BillYou Confess Don’t ConfessConfess (8, 8) (0, 15)Don’t Confess (15, 0) (1, 1)

Payoff matrix: each cell is a pair of payoff, the number of years in prison.

– (8,8) è both you and Bill go to prison for 8 years

If both you and Bill confess, you both go to prison for 8 years

If you confess but Bill doesn’t confess, then you are free while Bill goes to prison for 15 years

If both you and Bill remain silent, then you both go to prison for 1 year

Clearly the best result: both keep silent. (Q: Is this doable?)

Prisoner’s Dilemma – shall you confess or not?


You reason as follows:

• If Bill confess, then you intend to confess to get 8 years instead of 15 years prison

• If Bill does not confess, then you intend to confess to get 0 years instead of 1 year in prison

• Conclusion: you will confess!

Prisoner’s Dilemma


• Same as you, Bill will also confess! Both get 8 years in prison

• Dominant strategy: for both, confession is a dominant strategy that yields a better outcome regardless of the opponent’s choice

Dominant strategy

Nash Equilibrium (NE)

Nash Equilibrium (NE): A combination of strategies is called a Nash Equilibrium if neither player has an incentive to change strategy, given the other player’s choicemutual best response.

Best response: the strategy which produces the most favorable outcome, taking other players' strategies as given John Nash, Jr. (1928-‐2015)

Both confess is a Nash Equilibrium

Both don’t confess is not a Nash Equilibrium. Why?

Both don’t confess is not a Nash Equilibrium


If both do not confess , the rival will always want to deviate

Bill will confess, then be in prison from 1 year to zero

Equilibrium need not be efficient. Non-‐cooperative equilibrium in the Prisoner’s dilemma results in a solution that is not the best possible outcome for the parties.

Individual’s best choice is not the group’s best choice. An individual’s rational choice may lead to group’s non-‐rational choice

Paradox

Cooperative solution: They could both have been better off if they had reached the cooperative solution

What would you and Bill decide if they could negotiate?

That is exactly why police interrogate suspects in separate rooms

Recall “Battle of the Sexes” Game

Two equilibria: both play computer game; both go shopping.

Your girlfriend(Computer game)

Your girlfriend(Shopping)

You (Computer game) (10,5) (3,3)

You (Shopping) (0,0) (5,10)

However, either solution is unfair for one of you. (What is the equilibrium?)

Mixed Strategy

Two types of strategies: a pure strategy and a mixed strategy

A pure strategy: at every stage in the game, it specifies a particular move with complete certainty.

A mixed strategy: applies some randomization to at least one of the moves. The randomization is a set of fixed probabilities, where the sum of the probabilities is 1.

Outline

24



Discussions

DEMAND RESPONSE MANAGEMENTPricing Scheme

Energy balance: power generation is equal to power demand

S: power supply from generators

D: power demand from customers

Energy balance point: S=D

Energy balance is very important for energy systems stability and economy D

S

S>D

S<D

Market

clearin

g price P*

S=D

Demand Response Management (DRM) is the main approach to achieve energy balance

DRM studies the interaction between the supply and the demand sides by two-‐way flows of power and information.

A typical residential household load during winter

The power load may significantly vary over time and location

The practical load profile is very unbalanced

– Residential peak load (late afternoon)

– Industrial peak load (morning)

Household load in winter (source: www.vreg.be)

Peak hours are the main reason for power blackout

Different users have different power demand load

Residential users

Commercial users

Industrial users

The peak load issue

Power infrastructure is designed for peak loads.

Peaks have less than 1% of the time. Reducing peaks can then reduce power generation and save a lot of money

Capacity

Nordic u

sage per type

(10^9 KWh)

Source: Nord Pool Spot

Demand Response Management (DRM) definition

US. Department of Energy

Demand Response Management (DRM) is defined as changes in electric usage by end-‐use customers from their normal consumption patterns in response to changes in the price of electricity over time, or to incentive payments designed to induce lower electricity use at times of high wholesale market prices or when system reliability is jeopardized.

Users will change energy usage behaviors according to different electricity prices, or incentive payments, or system reliability

Demand Response Management: Objectives

No DRM

with DRM

Reduce energy consumption

§ encourage energy-‐aware consumption patterns

§ Reduce power generation

Shift the energy consumption

§ Mitigate power load during the peak hours

§ Improve grid reliability

Direct Load Control (DLC)

Intelligent Load Control / Pricing

A. Direct load control

B. Intelligent load control

Demand Response Management: Approaches

Intelligent Load Control / Smart Pricing

Price-‐based program provides users with different prices at different times

When users know about the price changes§ They will naturally use less electricity when electricity prices are high§ This will reduce the demand at peak hours

This program indirectly enforces users to dynamically change their energy usage patterns according to the variance of electricity prices, instead of directly controlling their loads.

Pricing models§ Time-‐of-‐Using (ToU) pricing; § Real-‐time Pricing (RTP); § …..

Time-‐of-‐Use (ToU) Pricing

When users consume energy at different time intervals of a day, they are charged at different electricity prices

ToU pricing is usually released far in advance, and keeps unchanged for a long time period.

§ on-‐peak price § mid-‐peak price § off-‐peak price

Examples

§ Ontario, Canada

§ Ausgrid (Australia)

Real-‐Time Pricing (RTP)

The electricity price usually varies at different time intervals of a day (e.g., in each hours)

RTP is usually released on an hour-‐ahead pricing or day-‐ahead pricing basis

Examples

• Chicago uses hourly-‐based RTP

• In Oslo, you may have RTP from Sognekraft AS called “Innkjøpspris” plan (source: http://www.strompris.no)

DEMAND RESPONSE MANAGEMENTHome Energy Management System

When shall we use appliances/devices at home?

time0 23126 18

In a house, a smart meter can automatically coordinate appliances to satisfy the user’s need via ON/OFF control

Power infrastructure

Communications infrastructure

Energy Consumption Scheduling: A simple example

Dishwasher after lunch (Q: why choose time period between 1:30-‐3:30pm?)

works at the lowest price between “Setup time” and “Deadline”

• Shiftable appliances (e.g., washing machines, Electric Vehicles, and clothes dryers): the operation task can be shifted to a different time period

Energy Consumption Scheduling is actually not simple since we have many different appliances

• Non-‐shiftable appliances (e.g., TV, lights, cooking): must be kept ON for a certain period of time.

• Q: which type of appliances?

refrigerator dishwashers fire alarm

Energy Consumption Scheduling problem

• Q: Given the price values, how should we schedule the power load?

• Scheduler should analyze

– User’s energy consumption needs

– Price values • The schedule is normally an optimal solution with different preferences

–Minimize the cost of electricity

–Maximize user’s comforttradeoff

• For each appliance 𝒂 ∈ 𝑨, we define a daily energy consumption scheduling vector 𝒙𝒂 as follows:

– where 𝑯 = 𝟐𝟒 hours

Energy Consumption Scheduling – parameters

• Let 𝑨 denote the set of appliances

–Washing machines, TV, lights, Dryer, Dishwasher, EVs (Electric Vehicle)…

𝒙𝟓𝟏𝟕 = 𝟔

𝒙𝟏𝟕 =2

02468Load

time

Energy Consumption Scheduling – operation time constraint

• For each appliance 𝒂 ∈ 𝑨, the user should indicate:

– 𝜶𝒂: beginning of the operation time (setup time)

– 𝜷𝒂: end of the operation time (deadline)• Operation should be scheduled within [𝜶𝒂, 𝜷𝒂]

Example

• Dish washer after lunch: setup time 𝜶𝒂= 12 Noon and deadline 𝜷𝒂= 6 PM (make dishes ready for dinner)

Energy Consumption Scheduling – power level constraint

• Each appliance 𝒂 ∈ 𝑨 usually has a maximum power level 𝜸𝒂𝒎𝒂𝒙

– NISSAN Leaf: charged up to 3.6 kW per hour

• Each appliance 𝒂 ∈ 𝑨may also have a minimum power level 𝜸𝒂𝒎𝒊𝒏

4 ≤ 𝒙678 ≤ 8

02468Load

time

• For each appliance 𝒂 ∈ 𝑨 , it is required that

𝜸𝒂𝒎𝒂𝒙 = 𝟖

𝜸𝒂𝒎𝒊𝒏 =4𝛾<=>? ≤ 𝑥<A ≤ 𝛾<=<B∀𝑎 ∈ 𝐴, ℎ ∈ [𝛼<,𝛽<]

• Given parameters 𝑬𝒂, 𝜶𝒂, and 𝜷𝒂, it is required that

Energy Consumption Scheduling – total energy constraint

• Let 𝑬𝒂 denote the total energy needed for the operation of appliance 𝒂 ∈ 𝑨

– Bosch WAS20160UC washing machine: 𝑬𝒂 = 0.36 kWh per load

𝒙𝟓𝟏𝟐 + 𝒙𝟓𝟏𝟑 + 𝒙𝟓𝟏𝟒 + 𝒙𝟓𝟏𝟓 + 𝒙𝟓𝟏𝟔 + 𝒙𝟓𝟏𝟕 =32

02468Load

time

Cost Minimization

Energy Consumption Scheduling problem to minimize cost:

– where 𝒑𝒉 denotes the price of electricity at hour h. Could be ToU or RTP model

Subject to~~~~~

Load from all appliances in hour h

ENERGY MARKETGame theory for the

Energy market players – power grid operators

Transmission system operators (TSOs) The TSO operates the transmission assets and is responsible for the power balance on the transmission system, e.g., Statnett

Distribution system operators (DSOs) Power distribution system to operate the distribution grid and transmit electricity to residential customers, e.g., Hafslund

Energy market players – sell and buy

Generating company (genco) The generators own production assets, whose generation is offered through the electricity market.

RetailerThe retailer buys electricity from the electricity market, then sell to the end-‐consumers.

CustomersThose eventually use the electricity for any purpose (from watching TV to heating to industrial production processes). There is a difference between small and large consumers, since the latter ones may be allowed to directly participate in the electricity market.

Energy market players – rule and operate the game

RegulatorsRegulators effectively ‘police’ the energy market. The regulator is responsible for the market design and its specific rules. It also monitors the market in order to spot misbehavior in electricity markets (collusion, abuse of market power, etc.).

-‐ NVE (The Norwegian Water Resources and Energy Directorate)

Market operatorPower exchange platform used by market players to negotiate purchases and sales of electricity., e.g., Nord Pool

Relationship between market players

Source: Kirschen and Strbac (2004). Fundamentals of Power System Economics]

How does electricity market work? – an example

An electricity generator bids an amount of production and a price they will sell

Generators submit thier bids simultaneously to the market in the form: (production, bid price)

Example: Generator 1 with bid (100, 10)

– Generator 1 wants to sell 100MW with price 10$/MW

Generator 1(100,10) Generator 2

(100,20)Generator 3 (100,30)

Demand

market clearing price

Market Clearing Price

Three generators send bidding proposals.

– Generator 1 bid (100; 10)



The Market Operator (e.g., Nord Pool) organizes the proposals by ascending order of prices Generator 1

100 200 300

Price ($/MW)

10

20

30

Deman

d170 MW

Generator 2

Generator 3

Production (MW)

The market operator finds the intersection between the demand line and the supply curve. This intersection gives the market clearing price.

Then, market clearing price is 20$/MW. This price is same for all generators when they are accepted and sell electricity.

Profits

The accepted bids are the ones in the left side of demand line:

– Generator 1 produces 100MW

– Generator 2 produces 70MW

– Generator 3 is not accepted

100 200 300

Price ($/MW)

10

20

30

Deman

d170 MW

Production (MW)

Generator 1 will produce 100MW with price 20$/MW

– The profit is: 100*(20-‐10)=1000

Generator 2 will produce 70MW with price 20$/MW

– The profit is 0

Generator 1

Generator 2

Generator 3

Electricity Market Strategies

Electricity producers play the game using their price and production. By changing these two parameters, the market sharing changes.

Cournot strategy: Two firms compete simultaneously on the quantity of output they produce of a homogeneous good.

Bertrand strategy:Two firms compete simultaneously on the price of a homogenous good.

Antoine Augustin Cournot (1801-‐1877)

Joseph Louis François Bertrand (1822-‐1900)

Cournot Model

Non-‐cooperative: Generators are non-‐cooperative, independently decide their production (simultaneously).

Rational and selfish: A generator should decide how much electricity to produce in order to maximize its profit without knowing the decision of the others.

Electricity price will be determined by the demand curve and supply curve where the total supply is equal to the total demand.

Cournot Game with Two Generators

Each generator chooses only between two levels of production

– High production: 75MW

– Low production: 20MW

The low production may be interpreted as withholding of capacity with a motivation to increase prices. If prices increase sufficiently, the generator can make a higher profit at the low production.

Generator ACost: 10$/MWhHigh: 75MWLow: 20MW

Demand

Generator BCost: 10$/MWhHigh: 75MWLow: 20MW

Cournot Game with Two Generators

Market price is set by the Market Operator

– If total power demand < 50MW, the price will be set as 150$/MWh

– If 50 < power demand < 100, the price will be set as 45$/MWh

– If 100 < power demand < 150MW, the price will be set as 40$/MWh

50 100 150

Price ($/MWh)

4045

150

Production (MW)

95For example, when the total power demand is 95MW, the price is set as 45$/MWh

Goal: to choose the power production level (either High or Low production) that maximizes their profits.

Power Production Matrix

For (High, High) = (75, 75), the total production is 75+75=150. According to the price curve, the price is $40.

PRODUCTION Generator BGenerator A High LowHigh (75, 75) (75, 20)Low (20, 75) (20, 20)

For (High, Low) = (75, 20), the total production is 95. According to the price curve, the price is $45.

For (Low, High) = (20, 75), the total production is 95. According to the price curve, the price is $45.

For (Low, Low) = (20, 20), the total production is 40. According to the price curve, the price is $150.

Nash equilibrium

PRICE Generator BGenerator A High LowHigh 40 45Low 45 150

Nash Equilibrium

PROFIT Generator BGenerator A High LowHigh (2250, 2250) (2625, 700)Low (700, 2625) (2800, 2800)

NEWhen both generators choose low levels of production to maximize their profits. No one has an incentive to change strategy, given the other player’s choice (mutual best response).

How to maximize revenues for multiple providers while optimizing consumer demands?

Research Problem

62S. Maharjan, Q. Zhu, Y. Zhang, S. Gjessingand T. Basar, "Dependable Demand Response Management in the Smart Grid: A Stackelberg Game Approach," in IEEE Transactions on Smart Grid, vol. 4, no. 1, pp. 120-‐132, March 2013.

Consumer side analysis: utility maximization

Problem formulation

K providers, N consumers:

Provider side analysis: revenue maximization

cost limit of user nunit price, available power of provider k

user parameters demand of user n from provider k

63

Proposed Stackelberg game model

Components of the Stackelberg game

Players

Strategies

Payoff

Leaders: providers, Followers: consumers

Providers: unit price, consumers: demand

Providers: revenue, consumers: utility

Stackelberg game framework

The payoffs of providers and consumers are coupled

Game theory to model the interactions

64

Providers

Users

1 K2

1 2 N

{yk, k � K}{xn, n � N}

Stackelberg equilibrium: Closed form solutions

Matrix A

where

A unique NE exists in the pricesetting game, thereby a unique SE.

Theorem 2

Matrix A is invertible and the solutions are positive

Theorem 1Consumer demands at Stackelberg equilibrium (SE)

Unit prices set by providers at SE

65

xn,k =Cn + �n

Pk�K yk

Kyk� �n

Numerical results

66

The increase in a consumer’s cost limit increases the unit prices too

The demand of user 1increases as does its cost limit

All other users are affected, thus, they get less power

Games

Stochastic games

Evolutionary games

…..

Non-‐cooperative vs. cooperatives games

One-‐shot games vs. Repeated games

Simultaneous moves vs. Sequential moves games

What type of game is a Stackelberg game?

Games with complete vs. incomplete information

Games with perfect vs. imperfect information

Zero-‐sum vs. non-‐zero sum games

Outline

68



Discussions

DISCUSSIONSQ&A

Setting Electricity Price

Oslo has two power companies A and B to provide electricity

A and B set different electricity price to attract customers

Any user can freely use either A or B

How can A or B decide the electricity price?

A B

If both generators cooperate, they can both charge the monopoly price. However, each generator has an incentive to reduce its price slightly and capture more market share, even though it knows that both generators will be worse off if they both cut price.

Structure similar to that of the prisoner’s dilemma.

Demand response with solar power and wind power

DRM with solar panel and wind power is difficult to model

Smart grid

…

Smart grid

…DRM with energy storage

Challenges?

Which game theory model should be used?

time0 23126 18

Demand response applications: Data Centers

• Daily services supported by data centers

– Gmail

– Facebook

– Dropbox

– DNB (supported by Green Mountain data center in Stavanger)

– Q: more?

Google Data Centre at Mayes County, Oklahoma


Why place data centers in Finland or undersea? Google data center

in Finland

Microsoft undersea data center

Data centers are huge energy consumers, in particular cooling systems, and

– pay a lot for electricity bill

–make power grid instable during peak hours

Cooling system uses sea water from the Bay of Finland and reduces energy use


Can DRM help data centers to reduce energy cost?

Allocate computation tasks to locations with cheaper prices

Google data centers worldwide

Oklahoma: 406kr/MWhFinland: 241kr/MWh

Singapore: 1450kr/MWh

10 tasks

2 tasks

4 tasksMorocco

76

References

T. Başar and G. J. Olsder, Dynamic Noncooperative Game Theory, ser. SIAM Series in Classics in Applied Mathematics. Philadelphia, PA: SIAM, 1999.

H. Singh, “Introduction to game theory and its application in electric power markets”, IEEE Comp. Applicat. in Power, vol. 12, pp. 18-‐22, Oct. 1999

S. Maharjan, Q. Zhu, Y. Zhang, S. Gjessing and T. Basar, "Dependable Demand Response Management in the Smart Grid: A Stackelberg Game Approach," in IEEE Transactions on Smart Grid, vol. 4, no. 1, pp. 120-‐132, March 2013.

R. Deng, Z. Yang, M. Chow and J. Chen, “A Survey on Demand Response in Smart Grid: Mathematical Models and Approaches,” IEEE Trans. Industrial Informatics, vol. 11, no. 3, June, 2015

Thank you!

Demand Response Pilot Study -‐ Findings on Users Behavior

The largest portion of peak load reduction is attributable to commercial & industrial customers and wholesale market participants. The growth of peak load reduction is mainly driven by those customers.

Relatively, residential customers make a modest contribution to peak load reduction.Chen and Liu, “From demand response to transactive energy: state of the art”, J. Mod. Power Syst. Clean Energy (2017), 5(1): 10-‐19

sabita maharjan energymarket and gametheory slides · 2018. 9. 19. · cournot,bertrand&...

Documents