TEA Connected Analytics JEA Zero Consumption Pilot Project

William Clarke, Executive Dir., Strategic Innovations & Analytics, TEA

Dr. Jian Hu, Research Scientist, Strategic Innovations, TEA

Kent Mathis, Manager of Utility Analytics, JEA

Today’s Discussion Topics

• What is Connected Analytics? (Bill)

• Description of the JEA Zero Consumption Pilot Project (ZeroCon) problem statement (Kent)

• Machine Learning and how it was used to solve the ZeroCon problem (Jian)

• Success! (Kent/Jian)

• What’s the future? (Bill)

2

TEA’s Five-year Strategic Priorities

3

0

Cost and Efficiency

Culture of Performance

Organic Growth

Emerging Services

Industry Solutions

Strategic Growth

TEA will control costs and improve

efficiencies to increase value.

TEA will promote a performance-based culture that inspires

innovation and excellence.

TEA will add new clients and market existing

services to community-owned

entities.

TEA will explore, create and expand services that add value to clients.

TEA will create value through structured collaboration and

coordination with clients.

TEA will evaluate strategic opportunities

to expand market reach or acquire resources.

Foundational Priorities Building Priorities

Connected Analytics Enablers/Drivers

• Modestly sized data sets

– Appropriate for economies-of-scale approach

• Existing Analytics Group

– Easily expanded to encompass the analysis of large smart meter volume data

• Trust relationship between Members

– enables the sharing of market and operationally sensitive data

4

Utility Analytics Value CurveV

alu

e

Time

OMG!Geez we have a

lot of data!

Data Fortress

Data stored, secured and

available

Basic ReportingAnswers “what

happened?” BUT not intuitive, limited

presentation

Business Intelligence

Dashboards, dynamic data use.

Answers “what happened?” now

more intuitive

PredictiveModeling and

planning based on historic data

ExecutionLeveraging real and near real-time data

Business

TransformationBusiness process change

initiated by analytics-derived information

1

2

3

4

5

6

7

Prescriptive Analytics

Predictive Analytics

Reporting Analytics

5

Solution Design Components

6

Data Scientists & Analysts

eSmart Systems

7

eSmart Systems

8

Today’s Discussion Topics

• What is Connected Analytics?

• Description of the JEA Zero Consumption Pilot Project (ZeroCon) problem statement

• Machine Learning and how it was used to solve the ZeroCon problem

• Success!

• What’s the future?

9

One example of using smart meter (AMI) data to improve operations and recover lost revenues

JEA ZERO CONSUMPTION PROJECT

10

JEA Overview

• JEA (formerly Jacksonville Electric Authority)• Originally a municipal ‘electric’ agency that started in 1895 • In 1997 JEA took over water and sewage for the city • About 2,300 employees

• Service Territory• 900 square miles• Four counties in NE Florida• Electric, Water, Sewer, Reclaim Water, Natural Gas, Chilled

Water

• 8th Largest Municipal in the US *(APPA, Customers Served and Electric Revenue)

150,000

200,000

250,000

300,000

350,000

JEA Service Territory

FY16FY15 FY15 FY16 FY15 FY16

443,705 451,788

325,

352

333,

139

250,

974

257,

719

Average Number of Customer Accounts

Electric System Water & Sewer System

Water

Sewer

Trade, Transportation

& Utilities 21%

Professional & Business Services

15%Education & Health Services

15%

Leisure & Hospitality

13%

Total Government

11%

Financial Activities

9%

Construction 6%

Other Services 5%

Manufacturing 5%

Source: “Florida Nonagricultural Employment – Most Recent 12-Months”

• Located in Northeast Florida, the Jacksonville Metropolitan Statistical Area (MSA) has an estimated population of 1.4 million according to the Census Bureau’s latest estimate1

• The Jacksonville MSA saw a 7.9% increase in population from April 1, 2010 to July 1, 20151

• Service territory also includes a small number of customers in neighboring St. Johns, Nassau and Clay Counties

• The local economy is made up of a diverse mix of industries

1 U.S. Census Bureau, Population DivisionAnnual Estimates of the Resident Population as of July 1, 2015

250,000

300,000

350,000

400,000

450,000

443,

705

451,

788

“Highest Customer Satisfaction with Business Electric Service in the South among Midsize Utilities.”

JEA received the highest numerical score among 13 midsize utilities in the South in the J.D. Power 2016 Electric Utility Business Customer Satisfaction Study, based on 21,852 responses, and measuring the experiences and perceptions of business customers surveyed March and November 2015. Your experiences may vary. Visit jdpower.com

JEA Electric System Overview

• Net Assets In-Service: $3.1B

• Power Production Assets

‒ 6 Plants, 18 Units

‒ Net Capacity: 3,414 MW (3,743 MW winter)

‒ Fuel Sources: Oil, Natural Gas, Coal, Petroleum Coke

‒ Small amount of renewables (solar, landfill gas)

• Transmission System

– Voltage Levels (KV): 500, 230, 138 & 69

– 745 Miles of Transmission

– 74 Substations; 200 Transformers (high side >= 69kV)

• Distribution System

– Voltage Levels (KV): 26.4, 13.2 & 4.16

– 336 feeders (214 – 26.4kV; 95 – 13kV; 27 – 4kV)

– 6,760 circuit miles (45% Overhead; 55% Underground)

– 102,600 transformers, 200,900 poles

1 The average dispatch prices at maximum load for each unit from 10/1/2015 through 9/30/20162 NS CFB burned a blend of pet coke and coal during FY16

FY16 Dispatch Stack1

Cap

acit

y (M

W)

0

500

1,000

1,500

2,000

2,500

3,000

3,500

Scherer 4 (Coal)$24.95/MWh

Combined Cycle (Gas)$19.85/MWh

NS CFB (Pet Coke & Coal)2

$22.90/MWh

SJRPP (Coal)$33.12/MWh

Simple Cycle CT (Gas)$27.14/MWh

NS3 (Gas)$27.52/MWh

Water & Sewer System Overview

Water System• 19 major and 18 small water treatment plants and

two re-pump facilities

• 134 active water supply wells, 4,449 miles of water distribution mains and total finished water storage capacity of over 70 million gallons

• Two major and four small distribution grids

Sewer System• Approximately 3,898 miles of gravity sewers and

force mains

• 1,375 pumping stations, 790 low pressure sewer units, and 11 treatment plants currently ranging in rated average daily treatment capacity from approximately 0.2 to 52.5 MGD

The Issue

From January 2011 through February 2016, JEA Water Meter Services team completed 30,316 truck rolls related to Zero Consumption. Of those truck rolls, 24,195 were “wasted” truck rolls because no work was needed. This translates to approximately $1.1M expense related to unnecessary truck rolls.

The project team consisting of JEA, TEA & eSmartinvestigated the historical truck rolls and developed an algorithm to better identify broken meters. The goal was to reduce wasted truck rolls to 20% or less.

Problem Statement

• Leverage AMI data to reduce the number of wasted truck rolls for suspected broken water meters from 80% to 20%

16

Current StateFuture State

Understanding the Problem: Why is the Meter Showing No Consumption?

No usage for 3 months in a row

Meter dead/brokenMeter stuck or in

process of breakingOccupied but no

water consumptionVacant

Meter dead/broken: normal water usage before the zero-con, normal electricity usage

Meter stuck or in process of breaking: normal/intermittent water usage before zero-con, normal electricity usage

Vacant: low electricity usage & variance, lots of zero water reads, high turnovers (rental?)

Occupied but no water consumption: no water usage (e.g. well in backyard) with normal electricity usage

Current ‘Broken Water Meter’ Process

Monthly Billing Reads

Create truck roll ticket

Working Properly?

List meter as working

Remove from report

for 1-yr

Repair/replace meter

Route to billing dept.

Repeat process

next month

>90 days Zero-con?

Billing Corrections?

N N

N

Y Y Y

Y

N

Y

Electric service at location?

>300 kWh at

location?

N

Pilot Objectives

• JEA and TEA would pursue two distinct objectives:

– Improve the accuracy of identifying zero consumption meters

– Determine if TEA adds value by providing this analytics service

19

Project Timeline

20

Project Kick Off

June July Aug Sept Oct Nov Dec

Data Collection

Analysis & Modeling

Initial Field Testing

Model Enhancements

Final Field Testing

Today’s Discussion Topics

• What is Connected Analytics?

• Description of the JEA Zero Consumption Pilot Project (ZeroCon) problem statement

• Machine Learning and how it was used to solve the ZeroCon problem

• Success!

• What’s the future?

21

A Supervised Machine Learning Approach

• Data gathering: get all data in one place, past truck rolls, water/electricity consumptions, billing/account data, etc.

• Data Engineering: Re-construct data scenarios around the truck rolls, identify featuresand statistical correlations

• Model Building: build and cross-validate the model

• Field testing and Deploy

22

Sample Statistical Studies

23

Corr(broken, elec_mean_12m) ~0.47 Corr(broken, turnovers) ~ -0.178 Corr(broken, meter_age) ~0.04

Machine Learning Models

• Decision Forest

• Supporting Vector Machine

• Neural Network

24

Model Building

25

• We leverage Microsoft Azure ML Studio for model building and cross-validation

• Model is exposed as a web API for integration

26

Precision = TP/(TP+FP) (% marked meters are truly dead)Recall (or TPR) = TP/(TP+FN) (% of dead meters marked)FPR = FP/(FP+TN)F1-Score = 2 * precision * recall / (precision + score)

Cross-Validation - NN vs SVM

NN Precision/Recall

27

Precision: % marked meters are truly deadRecall: % dead meters markedF1-Score = 2 * precision * recall / (precision + score)

System Integration

• Logic is exposed as a web API integrated in eSmart connected grid

• ML job can be scheduled on daily or weekly basis

28

Today’s Discussion Topics

• What is Connected Analytics?

• Description of the JEA Zero Consumption Pilot Project (ZeroCon) problem statement

• Machine Learning and how it was used to solve the ZeroCon problem

• Success!

• What’s the future?

29

Field testing - Ready to Go

Field TestingOne worker & Six Supervisors

Final Field Testing

• December 8-10, 2016

• Visited 102 meters by three different crews

• Correctly identified 89 broken meters

• 87% accuracy• Exceeded project goals

32

Recap• This project consumes lots of JEA smart-meter data

• Built on top of Microsoft Azure and eSmart Connected Grid, used both open source and proprietary data science technologies

• Finished in six months

– Adopted agile methodology - fail fast, learn quickly

– Successful collaboration between TEA, JEA and eSmart (Norway) across three time zones

• Two field testings conducted

– 89/102 new meters were identified correctly in 12/6 field trip —> 87% accuracy

– Reduction of Unnecessary Truck rolls > 90%

33

Overall Benefits to JEA

• Reduced wasted truck rolls

– Field crews redirected to other priorities

– Morale improved

• Uncovered unknown meter issues

• Revenue recovery

• Framework for additional uses cases

– Data access & security

34

Financial Benefits to JEA

• One-time Benefit: ~$180K– Over 4,000 zero consumption meters prior to year 2016.

Estimated that 10% (400 meters), can be recovered

• Annual Revenue Gain: ~$220k– 150 meters per month recovery for water & sewage revenue

• Annual Cost Avoidance: ~$270K– 72% truck roll reduction in total truck rolls (5400/7500), or 90%

reduction in unnecessary truck rolls (5400/6000)

Net value to JEA over three years: ~ $1.3 million

35

Today’s Discussion Topics

• What is Connected Analytics?

• Description of the JEA Zero Consumption Pilot Project (ZeroCon) problem statement

• Machine Learning and how it was used to solve the ZeroCon problem

• Success!

• What’s the future?

36

What’s Next?

• Well-formed, scalable structure • Business Plan

– Expand services to Members & Partners– Joint marketing with eSmart Systems

• Project Exploration • Fraud detection • Asset management• Business information dashboards• Long-term load forecasting• Revenue enhancement• Improved utility customer interaction• Utility planning & operations

37

Questions

38

Potential Use Cases – SideBar –

Potential Use Cases - Corporate• Business Information Dashboards

(AaaS)– Management metric dashboard– Customer dashboard– Municipal dashboard/portal– Operations dashboard

• Long-term load forecasting (AaaS)– Load forecasting by customer class– Residential load shape

segmentation– Residential engagement

segmentation– Solar forecasting & visualization

• Rates (AaaS)– Rate design by customer class– Rate validation– Pricing optimization– Cost & learn conservation program– Pricing optimization & rate design

• Revenue (AaaS)– Anomaly detection for meter

consumption– Revenue risk model– Fraud detection– Credit and collections

• IT (AaaS)– Information management

architecture– Data governance– Cyber/data security

40

Potential Use Cases – Customer Focus

• Customer engagement platform• Customer personal usage analytics• Customer profiles• Customer shut-off/turn-ons (Member function)• Customer payment systems• Interaction with customer premises (Nest technologies)• Using “Big Data” to understand and engage utility

customers (Member function)• Improving customer satisfaction with technology

applications & analytics designed to handle “Big Data”• Customer mobile• Customer segmentation

41

Potential Use Cases – Planning• Demand response (reduction) (AaaS)• Energy efficiency programs (AaaS)• Transformer load management (AaaS)• SCADA/OMS/AMI outage analysis (AaaS & Member function)• Voltage visualization (AaaS)• Distribution automation placement (AaaS)• The shift from time-based to condition-based maintenance (AaaS)• Workforce optimization (AaaS)• Managing transmission & distribution assets (AaaS)• Management of distributed generation (AaaS)• Resource expansion planning (AaaS)• Transmission & distribution asset planning (AaaS)• Inventory management (AaaS)• Equipment maintenance scheduling (AaaS)• Power quality optimization (AaaS)• Artificial Intelligence using 3-D imagery to view equipment in the field without physically being in

the field (AaaS)

42

Potential Use Cases• Meter health &

infrastructure– Inaccurate meter usage

(AaaS)– AMS system health (AaaS)– AMI executive dashboard

(AaaS)– Meter temperature data

analytics (AaaS)– Smart meter analytics –

health of the AMI system (AaaS)

– Meter communications infrastructure Operations (AaaS)

• Operations (Time sensitive solution)– Analytics for real-time

network operations (Member function)

– Estimated time of restoration (Member function)

– Mobile workforce (Member function)

– Storm restoration (Member function)

43

44

Problem Statement

• Question: when a water meter continuously reports 0, is it truly broken?

– 80% of the 30,000 zero-con truck rolls were ineffective ($50 per trip)

– Many broken meters were not be identified in-time due to limited field crews - can we stop the bleeding earlier?

• Project goal: can we reduce 80% of wasted trips to 20%?

45

Value

• ~$1.68M in three years

– ~$180k one-time

– ~$500k annually

• Main benefit components:

– Benefit of saved truck rolls ($50 per ~500 monthly)

– Early revenue recovery

– Non-monetary: Improved staff morale, customer satisfaction, environmental benefits

46