andy philpott epoc (epoc.nz) joint work with vitor de matos, ziming guan
DESCRIPTION
Advances in DOASA. Andy Philpott EPOC (www.epoc.org.nz) joint work with Vitor de Matos, Ziming Guan. EPOC version of SDDP with some differences Version 1.0 (P. and Guan, 2008) Written in AMPL/Cplex Very flexible Used in NZ dairy production/inventory problems - PowerPoint PPT PresentationTRANSCRIPT
EPOC Winter Workshop, October 26, 2010 Slide 1 of 31
Andy PhilpottEPOC
(www.epoc.org.nz)
joint work with
Vitor de Matos, Ziming Guan
Advances in DOASA
EPOC Winter Workshop, October 26, 2010 Slide 2 of 31
What is it?
• EPOC version of SDDP with some differences• Version 1.0 (P. and Guan, 2008)
– Written in AMPL/Cplex– Very flexible– Used in NZ dairy production/inventory problems– Takes 8 hours for 200 cuts on NZEM problem
• Version 2.0 (P. and de Matos, 2010) – Written in C++/Cplex– Time-consistent risk aversion– Takes 8 hours for 5000 cuts on NZEM problem
DOASA
EPOC Winter Workshop, October 26, 2010 Slide 3 of 31
Motivation
• Market oversight in the spot market is important to detect and limit exercise of market power.– Limiting market power will improve welfare.– Limiting market power will enable market
instruments (e.g. FTRs) to work as intended.• Oversight needs good counterfactual models.
– Wolak benchmark overlooks uncertainty – We use a rolling horizon stochastic optimization
benchmark requiring many solves of DOASA.• We don’t have access to SDDP. • We seek ways that SDDP can be improved.
DOASA
EPOC Winter Workshop, October 26, 2010 Slide 4 of 31Source: CC Report, p 200
Counterfactual 1The Wolak benchmark
EPOC Winter Workshop, October 26, 2010 Slide 5 of 31
What is counterfactual 1?
– Fix hydro generation (at historical dispatch level).– Simulate market operation over a year with thermal plant
offered at short-run marginal (fuel) cost.– “The Appendix of Borenstein, Bushnell, Wolak (2002)* rigorously
demonstrates that the simplifying assumption that hydro-electric suppliers do not re-allocate water will yield a higher system-load weighted average competitive price than would be the case if this benchmark price was computed from the solution to the optimal hydroelectric generation scheduling problem described above” [Commerce Commission Report, page 190].
(* Borenstein, Bushnell, Wolak, American Economic Review, 92, 2002)
The Wolak benchmark
EPOC Winter Workshop, October 26, 2010 Slide 6 of 31
Counterfactual 1What about uncertain inflows?
wet
dryStochastic program counterfactualThe optimal generation plan burns thermal fuel in stage 1 in case there is a drought in winter. The competitive price is high (marginal thermal fuel cost) in the first stage, but zero in the second (if wet).
Counterfactual 1In the year under investigation, suppose all generators optimistically predicted high inflows and used all their water in summer. They were right, and no thermal fuel was needed at all. Counterfactual prices are zero.
summer winter
EPOC Winter Workshop, October 26, 2010 Slide 7 of 31
Yearly problem represented by this system
S
N
demand
demandWKO
HAW
MAN
H
demand
EPOC Counterfactual
EPOC Winter Workshop, October 26, 2010 Slide 8 of 31
Cost-to-go recursion DOASA
EPOC Winter Workshop, October 26, 2010 Slide 9 of 31
DOASA: Cutting planes define the future cost functionDOASA
EPOC Winter Workshop, October 26, 2010 Slide 10 of 31
SDDP versus DOASADOASA
SDDP (literature) DOASA
Fixed sample of N openings Fixed sample of N openings
Fixed sample of forward pass scenarios (50 or 200)
Resamples forward pass scenarios (1 at a time)
High fidelity physical model Low fidelity physical model
Weak convergence test Stricter convergence criterion
Risk model (Guigues) Risk model (Shapiro)
EPOC Winter Workshop, October 26, 2010 Slide 11 of 31
p11
p13
p12
How DOASA samples the scenario tree
EPOC Winter Workshop, October 26, 2010 Slide 12 of 31
p11
p13
p12
How DOASA samples the scenario tree
EPOC Winter Workshop, October 26, 2010 Slide 13 of 31
p11
p13
p21
p21
p21
How DOASA samples the scenario tree
EPOC Winter Workshop, October 26, 2010 Slide 14 of 31
DOASA run times
0
2
4
6
8
10
12
0 500 1000 1500 2000 2500 3000 3500 4000 4500 5000
Number of forward passes through tree
Co
mp
uta
tio
n t
ime
(ho
urs
)
EPOC Winter Workshop, October 26, 2010 Slide 15 of 31
Why do it this way?Lower bounds converge faster
EPOC Winter Workshop, October 26, 2010 Slide 16 of 31
Why do it this way?Upper bound convergence: 5000 forward simulations
EPOC Winter Workshop, October 26, 2010 Slide 17 of 31
Takeaways
• In this case terminating SDDP after 4, or 5, or even 10 iterations (of 200 scenarios each) does NOT guarantee a close to optimal policy.
• Confidence intervals with 200 scenarios are 5 times bigger than with 5000 scenarios.
• Single forward pass is better as it does not duplicate cut evaluation.
• Iterations slow down as cut sets increase. Cut-set reduction needed.
SDDP
EPOC Winter Workshop, October 26, 2010 Slide 18 of 31
Rolling horizon counterfactual
– Set s=0– At t=s+1, solve a DOASA model to compute a
weekly centrally-planned generation policy for t=s+1,…,s+52.
– In the detailed 18-node transmission system and river-valley networks successively optimize weeks t=s+1,…,s+13, using cost-to-go functions from cuts at the end of each week t, and updating reservoir storage levels for each t.
– Set s=s+13.
Application to NZEM
EPOC Winter Workshop, October 26, 2010 Slide 19 of 31
We simulate an optimal policy in this detailed system
MAN
HAW
WKO
Application to NZEM
EPOC Winter Workshop, October 26, 2010 Slide 20 of 31
Gas and diesel industrial price data ($/GJ, MED)Application to NZEM
EPOC Winter Workshop, October 26, 2010 Slide 21 of 31
Heat rates Application to NZEM
EPOC Winter Workshop, October 26, 2010 Slide 22 of 31
Load curtailment costsApplication to NZEM
EPOC Winter Workshop, October 26, 2010 Slide 23 of 31
Market storage and centrally planned storage New Zealand electricity market
EPOC Winter Workshop, October 26, 2010 Slide 24 of 31
New Zealand electricity market
=(NZ)$12.9 million per year (=2.8% of historical fuel cost)
Estimated daily savings from central plan
EPOC Winter Workshop, October 26, 2010 Slide 25 of 31
Savings in annual fuel costTotal fuel cost = (NZ)$400-$500 million per annum (est)
Total wholesale electricity sales = (NZ)$3 billion per annum (est)
New Zealand electricity market
EPOC Winter Workshop, October 26, 2010 Slide 26 of 31
The next steps
How does risk aversion affect prices and efficiency?
How to model this? Use CVaR (Rockafellar and Urysayev, 2000)
Actually, need a time-staged version of this.
(Ruszczynzki, 2010), (Shapiro, 2010)
Application to NZEM
EPOC Winter Workshop, October 26, 2010 Slide 27 of 31
CVaR1-= Conditional value at risk (tail average)
Application to NZEM
0
0.0045
0 100 200 300 400 500 600 700
Annual fuel+shortage cost ($M)
90%
10%
VaR0.9= $420M
CVaR0.9= $460M
EPOC Winter Workshop, October 26, 2010 Slide 28 of 31
Average 2006 storage trajectories minimizing (1-)E[Z]+CVar(Z)
A risk-averse central planner
0
2000000000
4000000000
6000000000
8000000000
10000000000
12000000000
14000000000
16000000000
1 3 5 7 9 111315171921232527293133353739414345474951
Lambda 0
Lambda 0.5
Lambda 0.9
EPOC Winter Workshop, October 26, 2010 Slide 29 of 31
Total cost - residual water value
150 200 250 300 350 400 450 500 550
(NZ)$M
Lambda 0
Lambda 0.5
Lambda 0.9
“Fuel and shortage cost – residual water value” CDF
A risk-averse central planner
0
1
EPOC Winter Workshop, October 26, 2010 Slide 30 of 31
Conclusions
• DOASA is well-tested tool for benchmarking.• We now have a good empirical understanding of
convergence behaviour.• We can model risk aversion effectively.• Next steps
– include 2008-2009 inflow data– simulate central plans with different levels of risk aversion– How much risk can be avoided for $50M fuel cost?– Examine winter 2008 in more detail – especially price
outcomes.• Interested in feedback from participants – is this
worth pursuing? If so how should industry fund it?