Future Cities: Trinity Smart and Sustainable Cities Research Centre

Prof. Siobhán Clarke

Director

1

Trinity’s Future Cities Centre launched in July 2013

www.tcd.ie/futurecities/

Energy Demand-Side Management

3

Demand-Side Management

• Energy usage not distributed evenly during the day

• Morning peak, large evening peak, valley during the night

• Demand side management (DSM): modification of the consumers' electricity consumption with respect to their expected consumption

• DSM techniques – peak clipping, valley filling, load shifting …

• Based on prediction – influence consumers to defer loads that are not essential during the peaks and run them during low demand periods instead

• Can influence use of renewable too – defer loads during the periods of low availability etc

4

Demand-Side Management

5

Centralised: Evolutionary Algorithms in EV Scenario A schematic view of the problem used in this work with two main goals: • (a) that each EV is as

fully charged as possible at time of departure

• (b) reduce transformer load by avoiding charging more than one EV at a particular time, whenever possible.

6

Centralised: Evolutionary Algorithms in EV Scenario

Evolutionary Algorithms are search methods that take their inspiration from natural selection and survival of the fittest in the biological world.

1: Randomly create an initial population of individuals (a.k.a. candidate solutions). 2: repeat 3: Execute each individual and ascertain its fitness. 4: Select one or two individuals from the population with a probability based on fitness to participate in genetic operations (e.g., mutation, crossover, …). 5: Create new individuals by applying genetic operations with specified probabilities. 6: until an acceptable solution is found or some other stopping condition is met (e.g., a maximum number of generations is reached). 7: return the best-so-far individual.

7

Centralised: Evolutionary Algorithms in EV Scenario

Transformer load, averaged for 28 days, for 9 EVs with different initial state of charge (SoC).

All 9 EVs charged between 18:00 and 07:30. • Red squares shows the transformer average load for the greedy approach

(i.e., EVs start charging as soon as they reach home) • Blue circles show the average load using our proposed approach.

8

Centralised: Set Point Control Approach • Two types of demand:

base load

flexible load (e.g., electric vehicles, water heaters)

• Two set point algorithms to control

flexible load

Variable charging rate

Uses an EV charger that can vary its power (0-100%)

The transformer broadcasts the charging rate (0-100%) that each of the available EVs should charge at.

The feedback is the measured aggregate power demand at the transformer.

Variable connection rate

Uses a much simpler on-off type of charger

The transformer broadcasts the connection rate (0-100% probability) that each of the available EVs should attempt to connect at.

The feedback is the measured aggregate power demand at the transformer.

9

Centralised: Set Point Control Approach

0

100

200

300

400

500

600

700

800

0

20000

40000

60000

80000

100000

120000

140000

0:0

0

6:0

0

12

:00

1

8:0

0

0:0

0

6:0

0

12

:00

1

8:0

0

Vo

lt A

mp

s

Variable connection rate - set point 100 kVA

Total Load

Base Load

Connection Rate

0 100 200 300 400 500 600 700 800

0

20000

40000

60000

80000

100000

120000

140000

0:0

0

6:5

2

13

:44

20

:36

3:2

8

10

:20

17

:12

Vo

lt A

mp

s

Variable connection rate - set point 75 kVA

Total Load

Base Load

Connection Rate

0

200

400

600

800

0

20000

40000

60000

80000

100000

120000

0:0

0

9:3

6

19

:12

4:4

8

14

:24

Vo

lt A

mp

s

Variable charging rate - set point 100kVA

Total Load

Base Load

Charge Rate

0

100

200

300

400

500

600

700

800

0

20000

40000

60000

80000

100000

120000

0:0

0

4:4

8

9:3

6

14

:24

19

:12

0:0

0

4:4

8

9:3

6

14

:24

19

:12

Vo

lt A

mp

s

Variable charging rate - set point 75 kVA

Total Load

Base Load

Charge Rate

10

Centralised: Set Point Control Approach

• Shaping demand

• Enriched with urgency protocol with inherent backoff to ensure user utility

When the time to charge gets close to the time left (< 10 minutes difference), the device agent changes to the urgent state and starts to charge fully at each time step

• Set point control can be used to determine the bounds on transformer – if they are set too low utility might not be met and spikes occur

0

200

400

600

800

0

20000

40000

60000

80000

100000

120000

140000

0:0

0

8:0

0

16

:00

0:0

0

8:0

0

16

:00

0:0

0

8:0

0

16

:00

Vo

lt A

mp

s

100-80-60-80-100kVA control schedule

Total Load

Base Load

Control

Connection Rate 0

200

400

600

800

0

20000

40000

60000

80000

100000

120000

140000

0:0

0

8:0

0

16

:00

0:0

0

8:0

0

16

:00

0:0

0

8:0

0

16

:00

Vo

lt A

mp

s

Constant 75kVA

Total Load

Base Load

Control

Connection Rate

11

Decentralised approaches

12

Decentralised: Enhancements to demand side management in residential communities

• Target: Avoid high peak usage of electric appliances

• Short term load forecasting:

Estimate a residential community’s power demand ahead of time (day ahead, week ahead) based on historical load and weather information

Tackle issues of small scale (i.e. unpredictable human factor)

Detect anomalous changes from expected demand; match anomalies with previously encountered information and attempt re-prediction

13

Decentralised: Enhancements to demand side management in residential communities

• Normal days prediction:

Combines several techniques with various advantages (ANN, WNN, ARIMA, NF)

Focus on a community of 230 houses (half-hourly recorded demand from the CER smart-meter trial)

Uses historical weather information from Dublin airport station

Achieves 2.39% NRMSE (evaluation over 20 consec. days)

Monday Tuesday Wednesday Thursday Friday

14

Decentralised: Enhancements to demand side management in residential communities

• Anomalous days prediction: Detects anomalous

pattern changes in day demand as it progresses

Matches anomaly type from previously classified days and triggers re-prediction based on them

3.63% prediction error, 65% detection rate at 2:30pm

15

Decentralised: Multi-agent residential demand side management based on load forecasting

• Implement the grid as a multi-agent system - each EV is controlled by an RL-agent which implements 3 policies:

Policy 1: achieve at least the minimum required battery charge

Policy 2: charge at the minimum possible price/during the lowest load

Policy 3: keep under set transformer limits

• Agents given

Current load/ current price only

Predicted load/ predicted price too

16

Decentralised: Multi-agent residential demand side management based on load forecasting

• Uses reinforcement learning

• Distributed W-Learning (DWL) Multiple policies on each agents

Multiple agents collaborating

Learn dependencies between neighbouring agents

• Each agent learns how its actions affect its neighbours

17

Decentralised: Multi-agent residential demand side management based on load forecasting

• Simulations performed with 9 households (with EV agent + base load each)

• The required daily mileage differs ranges from 50 miles (requiring about 35% of full battery charge) to 80 miles (requiring about 50% of full battery charge). Charging decisions made every 15 minutes

• Base load taken from the data recorded in smart meter trial performed in Ireland in 2009-2010.

18

Decentralised: Parallel Transfer Learning • There are repeated patterns in smart grid

• Both over time and across the grid

• Dynamism in the grid can require relearning

• While learning performance tends to be bad

• Transfer Learning accelerates learning by providing initial information.

• Requires converged information prior to execution

• Information must be relevant to target task to accelerate learning

• Cannot capture the dynamism of policy relationships

19

Decentralised: Parallel Transfer Learning • Parallel Transfer Learning is an on-line version of Transfer

Learning Source and target tasks learn simultaneously, sharing information

whenever they deem it necessary

• This allows the relatedness of tasks to be exploited

• Multiple transfers allow dynamicity of inter-policy relationships to be shared

Transfer Learning Parallel Transfer Learning

20

Decentralised: Parallel Transfer Learning

• Using PTL reduces learning time to 1/3 of previous

21

Energy DSM Papers to date Conferences • E. Galvan-Lopez, A. Taylor, S. Clarke, and V. Cahill. Design of an Automatic Demand-Side Management System

Based on Evolutionary Algorithms . Proceedings of the 29th Annual ACM Symposium on Applied Computing, SAC '14, Gyeongju, Korea, March 24 - 28, 2014. ACM

• C. Harris, I. Dusparic, E. Galvan-Lopez, A. Marinescu, V. Cahill, and S. Clarke. Set Point Control for Charging of Electric Vehicles on the Distribution Network. . In Innovative Smart Grid Technologies (ISGT). IEEE PES, 2014.

• A. Marinescu, I. Dusparic, C. Harris, S. Clarke, and V. Cahill. A hybrid approach to very small scale electrical demand forecasting. In Innovative Smart Grid Technologies (ISGT). IEEE PES, 2014.

• Ivana Dusparic, Colin Harris, Andrei Marinescu, Vinny Cahill, and Siobhan Clarke. Multi-agent residential demand response based on load forecasting. In Technologies for Sustainability (SusTech), 1st IEEE Conference on, pages 90-96. IEEE, 2013

• E. Galvan-Lopez, C. Harris, I. Dusparic, S. Clarke, and V. Cahill. Reducing Electricity Costs in a Dynamic Pricing Environment. IEEE SmartGridComm, pages 169 - 174, Tainan City, Taiwan, 5 - 8 November, 2012, IEEE Press.

• C. Harris, R. Doolan, I. Dusparic, A. Marinescu, V. Cahill, and S. Clarke. A Distributed Agent Based Mechanism for Shaping of Aggregate, ENERGYCON 2014

• E. Galvan-Lopez, C. Harris, L. Trujillo, K. Rodriguez, S. Clarke and V. Cahill. Autonomous Demand-Side Management System Based on Monte Carlo Tree Search, ENERCYCON 2014

• A. Marinescu, I. Dusparic, C. Harris, S. Clarke, and V. Cahill. A Dynamic Forecasting Method for Small Scale Residential Electrical Demand, International Joint Conference on Neural Networks (IJCNN), IEEE, 2014

• A. Taylor, I. Dusparic, E. Galvan-Lopez, S. Clarke, and V. Cahill. Accelerating Learning in Multi-Objective Systems through Transfer Learning, World Congress on Computational Intelligence (WCCI), IEEE, 2014

Workshops • A. Taylor, I. Dusparic, E. Galvan-Lopez, S. Clarke, and V. Cahill. Transfer Learning in Multi-Agent Systems Through

Parallel Transfer. The 30th International Conference on Machine Learning, Atlanta, USA, 16 - 21, 2013. • A. Marinescu, C. Harris, I. Dusparic, S. Clarke, and V. Cahill. Residential electrical demand forecasting in very small

scale: An evaluation of forecasting methods. In Software Engineering Challenges for the Smart Grid, 2nd International Workshop on (SE4SG). IEEE, 2013

• A. Taylor, E. Galvan-Lopez, S. Clarke, and V. Cahill. Management and Control of Energy Usage and Price using Participatory Sensing Data. Eleventh International Conference on Autonomous Agents and Multiagent Systems, pages 111-119, Valencia, Spain, 2012.

Thank you.

http://www.tcd.ie/futurecities/

Prof. Mélanie Bouroche

Melanie.Bouroche@scss.tcd.ie

Trinity College Dublin