facilitating access to consumer electricity data · facilitating access to consumer electricity...

HoustonKemp.com

Facilitating access to consumer

electricity data

A draft report for the Department of Environment and Energy

February 2018

HoustonKemp.com

Report Authors

Adrian Kemp

Martin Chow

Contact Us

Sydney

Level 40

161 Castlereagh Street

Sydney NSW 2000

Phone: +61 2 8880 4800

Singapore

8 Marina View

#15-10 Asia Square Tower 1

Singapore 018960

Phone: +65 6817 5010

Disclaimer

This report is for the exclusive use of the HoustonKemp client named herein. There are no third party beneficiaries with respect to this report, and

HoustonKemp does not accept any liability to any third party. Information furnished by others, upon which all or portions of this report are based, is believed

to be reliable but has not been independently verified, unless otherwise expressly indicated. Public information and industry and statistical data are from

sources we deem to be reliable; however, we make no representation as to the accuracy or completeness of such information. The opinions expressed in

this report are valid only for the purpose stated herein and as of the date of this report. No obligations is assumed to revise this report to reflect changes,

events or conditions, which occur subsequent to the date hereof. All decisions in connection with the implementation or use of advice or recommendations

contained in this report are the sole responsibility of the client.

Facilitating access to consumer electricity data

HoustonKemp.com

Contents

Executive Summary i

1. Introduction 1

2. The impediments to accessing consumer’s electricity data 3

2.1 The right to access electricity consumption data 3

2.2 The need to do more to facilitate access to consumer data 3

2.3 Addressing the impediments 5

2.4 Vision for future consumer energy data access 6

3. A framework for facilitating access to consumers’ energy data 7

3.1 High-level framework to facilitate access to consumers’ energy data 8

3.2 Implementation of a consumer data access scheme 9

3.3 Questions for stakeholders 10

4. Verification of authorisation procedures 11

4.1 The current approach to verification of authorisation 11

4.2 What do retailers and distributors need to do to comply with the requirements of the

Privacy Act? 12

4.3 A proposed approach to verification – an accreditation scheme for third parties 14


5. What data should be made available? 17

5.1 The current data right and availability of consumers’ data 17

5.2 Assessing the merits of facilitating access to additional consumer data 18

5.3 Recommended scope of data to be provided to third parties 20


6. Handling requests and data transfer 21

6.1 Current process for providing data 21

6.2 Options for handling requests and data transfers 22

6.3 Preliminary recommendation 27


Facilitating access to consumer electricity data

HoustonKemp.com

A1. Summary of current arrangements for requesting and obtaining consumers’ metering data 28

A2. Summary of workshops held to date 30

Figure

Figure 1: Overview of existing data flows 22

Figure 2: Overview of a centralised data provision approach 23

Figure 3: Overview of a decentralised data provision approach 25

Tables

Table 1: Assumed establishment and ongoing cost of a centralised data provision approach 24

Table 2: Assumed establishment and ongoing cost of a decentralised data provision approach 26

Facilitating access to consumer electricity data Introduction

HoustonKemp.com i

Executive Summary

In 2016 electricity consumers, or their authorised representative, were given a right to access metering data

from distributors or retailers This was intended to empower consumers to make decisions about electricity

use, investments in solar PV and other energy savings technologies, and choices of competitive retailer with

a clear understanding of historic consumption patterns. At that time, it was believed that consumers would

most likely receive the benefits from accessing their data through third parties, who would offer advice

tailored to the specific circumstances of the consumer, given their consumption profile.

Much of the focus since 2016 has been on ensuring that customers have access to consumption data,

principally via portals provided by retailers. In contrast, there has been little attention to the processes and

systems needed to facilitate access by authorised third parties wishing to provide value added services to

consumers.

It is within this context that HoustonKemp Economists has been engaged by the Council of Australian

Governments (COAG) Energy Council to examine how, and make recommendations for, streamlining the

process, and facilitating timely access to consumers consumption data by authorised third party service

providers.

There is a need to develop a consumer electricity data access scheme

We have engaged with stakeholders representing distributors, retailers, consumers and some third-party

data users, and have identified several impediments to accessing consumers’ data by third parties, namely:

different assessments by distributors and retailers as to what is required to manage privacy risks and obligations, particularly due to concerns about reputational risks;

absent clear funding arrangements, strong incentives to minimise the cost of providing data, which leads to differences in data formats, changes in data formats over time, and different delivery mechanisms; and

incentives to minimise data provided due to concerns, particularly by retailers, that the information may be used to increase a competitor’s market share.

While in principle there is merit in the industry developing a common approach to facilitating access to data.

We believe that amending the National Electricity Rules (NERs) to require the Australian Energy Market

Operator (AEMO) to develop a consumer electricity data access scheme (the data access scheme) is the

best approach to facilitate on-demand access to consumers’ data by third parties. We believe that AEMO is

best placed to develop such a scheme, given its familiarity with existing data transfer processes and

procedures and its governance arrangements for registering market participants.

The proposed objective for the data access scheme is:

To facilitate on-demand access by retail customers or a customer’s authorised representative to

consumer electricity data.

The proposed consumer electricity data access scheme would set out:

the process and requirements for accreditation of customer authorised representatives, including information requirements and timeframes for accreditation;

provide a uniform approach to retail customer verification and consent procedures;

provide a centralised approach to the provision of metering data to customer authorised representatives;

specify a uniform form in which retail customers’ metering data must be provided to customer authorised representatives; and

specify the procedures for requesting and approving changes to the detailed data format by any party.


HoustonKemp.com ii

We are proposing that the scheme be developed by AEMO within 12 months of the rule changes giving effect to the scheme.

The scheme will have uniform procedures for the verification of authorisation

of a customer representative to access a consumer’s metering data

A key element of the proposed data access scheme is the need to develop uniform procedures for the

verification of authorisation of a customer representative to access a consumer’s metering data. The

challenge is how best to verify authorisation to satisfy risks of fraudulent use of consumers’ energy data,

while providing on-demand access to the data by a consumer’s authorised representative.

We believe that on-demand access can be provided whilst fulfilling privacy obligations by developing an

approach to verification that:

requires a third-party data seeker to be accredited prior to requesting data, to ensure that they have the necessary systems and processes to obtain and keep a record of a customer’s authorisation consistent with privacy obligations, and are seeking to provide services to consumers by accessing their data;

allows distributors or retailers to presume that an accredited third-party has obtained the necessary consumer consents when receiving a request for access to a consumer’s metering data; and

uses audit processes to ensure that fraudulent access to consumer’s energy data has not been obtained.

We expect that this approach to verification will allow data to be transferred more quickly to third parties, and

so will facilitate the on-demand access to consumer metering data.

Metering data to be provided will be consistent with existing data formats to

minimise initial set up costs while providing flexibility to evolve over time

Given uncertainties about the potential data needs by accredited third parties, we propose that the metering

data made available at the outset of the data access scheme be consistent with the Meter Data File Format,

specified by AEMO. The motivation for this, is to facilitate access to data as soon as practical, without

incurring undue time and expense in developing new or additional data formats.

That said, there may be merit in some incremental changes to be made to existing data formats should this

become apparent over the course of the development of the data access scheme.

Importantly, the data access scheme should also provide a process for any party to request changes to the

data made available, and its format. In determining whether to make changes to the format, AEMO should

consider the likely costs involved and whether the change can be expected to deliver outcomes that warrant

the cost. Nothing in this process should prevent commercial arrangements being put in place whereby

accredited third parties fund bespoke changes to standard data formats to facilitate the provision of services

to consumers.

AEMO should be given responsibility to establish the systems and processes

needed to provide access to authorised representatives

Our consultation process has highlighted that AEMO has been investing in data system upgrades to support

several changes to metering and market arrangements. Many of these changes will support the transfer of

data to third parties.

We have considered whether the systems and processes for the provision of data to third parties upon

request should be centrally managed, or undertaken by each distributor/retailer separately. Ultimately this is

a question of both cost and practicality.

In our opinion, there is merit in the data transfer process being centralised. This reflects our opinion that:


HoustonKemp.com iii

the overall cost of providing a centralised database, with associated API access by third parties is likely to be lower than if each distributor/retailer was required to build the necessary systems to deliver the same data service to third parties;

there are benefits from having a single point of access to data, for ease of access by those third parties that would otherwise need to seek data from multiple distributors/retailers using different systems; and

there are benefits from having only one system to change to implement system changes in responsive to innovations over time.

Our preliminary recommendation is for AEMO to be given responsibility to develop the proposed centralised

system, to maximise the opportunity to leverage existing systems. We believe this is a pragmatic approach

rather than reflecting any intrinsic reason that an electricity market operator should be responsible for

managing consumer access to metering data. It follows, that there may be merit in the future for such a

system to be outsourced to an independent third party so as to use competitive pressures to minimise the

development and ongoing operational costs, and given this service is not within the core functions of AEMO.

Questions for stakeholders

1. Is the proposed objective for the consumer electricity data access scheme appropriate?

2. Should AEMO or an alternative agency be given responsibility for developing the consumer electricity data access scheme?

3. Are there additional elements that the scheme should incorporate to facilitate access to consumers’ electricity data by authorised representatives?

4. What changes can be done in the short term without a rule change and what changes require a rule change to implement?

5. Are there alternative approaches to managing verification of consumer identity and third-party authorisation that should be considered and which are consistent with the scheme objective of providing on-demand access to data by authorised third parties?

6. Should AEMO or another agency be given responsibility for accrediting third parties?

7. Should authorised and accredited third parties be given access to more than just a consumer’s metering

data upon the commencement of the data access scheme?

8. What are the arguments for and against providing third party access to retail and/or network tariff data?

9. What changes are required to existing AEMO metering data formats to facilitate access by third parties

to consumer electricity data?

10. Are the estimated costs for development and ongoing maintenance a centralised or decentralised

implementation of the system reasonable?

11. What are reasonable timeframes for implementation under each of the options considered?


HoustonKemp.com 1

1. Introduction

A combination of advancements in electricity metering technology and greater opportunities for consumers to

manage their electricity use and generate their own electricity means that there is both a greater availability

of, and potential value from accessing, consumers’ energy data.

However, there have been impediments to consumers obtaining value from their own energy data. These

impediments have affected the scope and ease of access to a consumer’s electricity data, by both

consumers themselves, and by authorised third parties seeking to provide consumers with value-added

services.

Key amongst these impediments was the lack of a right for a consumer to access their own energy

consumption data, or authorise a third party to access the data. This impediment was alleviated by a change

to the National Electricity Rules (NERs) on 1 March 2016, which gave consumers or their authorised

representative a right to obtain metering data from distribution network service providers (DNSPs) or

retailers.

However, simply providing a right of access to consumers’ metering data has not led to the anticipated

increase in the use of the data, particularly by value adding authorised third parties. Part of the reason is that

there are other impediments to accessing the data, including (amongst other things):

complex arrangements put in place by distributors and retailers for authorised third parties accessing data, to manage privacy obligations;

lack of uniformity about data access arrangements, including processes for fulfilling privacy obligations, data formats, and data provision arrangements; and

the length of time from which a data request is made, and the data provided.

These access process related impediments impose costs on third parties wanting to provide value added

services to consumers, which likely contributes to the lack of growth in the provision of these services.

It is within this context that HoustonKemp has been engaged on behalf of the Council of Australian

Governments’ (COAG) Energy Council to examine how, and make recommendations for, streamlining the

process, and facilitating timely access to consumers’ consumption data to authorised third party service

providers, acting on behalf of consumers. We refer to this as a consumers’ electricity data access framework

for authorised third parties (electricity data access framework).

Our draft recommendations and conclusions have benefited from information gained through extensive

consultations with distributors, retailers, third party consumer service providers, and consumer

representatives. This involved conducting a series of workshops and bilateral discussions with stakeholders

to canvass the key issues, explore options for addressing impediments to data access by third parties, and

to collect information on current arrangements. This input has been invaluable to the formulation of our

preferred approach to achieving a vision of on-demand access to consumers’ energy data by authorised

third parties.

This project is occurring within a wider policy context where consumers are being given greater rights to

access their own data held by companies, including in the banking and telecommunications sectors. Many of

the matters that we have considered to facilitate third party access to consumer energy data have wider

relevance. In our opinion, there is merit in ensuring that elements of the data access framework for electricity

is consistent as a matter of principle with other data access frameworks that arise in other sectors. That said,

there are likely to be unique characteristics within any given sector that mean the specific approaches used

to facilitate access might differ. Where relevant, we consider these matters throughout this report.

The remainder of this report is structured as follows:


HoustonKemp.com 2

section 2 provides greater context for the project and describes the vision of access to consumers’ energy consumption data;

section 3 sets out our proposed framework for facilitating third party access to consumers’ electricity data, through the creation of a consumer electricity data access scheme;

section 4 sets out current verification and consent processes applying in the energy sector, and how those processes might be modified to achieve the vision of access to consumers’ energy consumption data;

section 5 discusses the scope of data that should be covered by the electricity data access framework; and

section 6 sets out the possible technology options for supporting the electricity data access framework, likely costs, and a preferred way forward.

Appendix A1 summarises current approaches to obtaining and providing consumers energy data.

Facilitating access to consumer electricity data The impediments to accessing consumer’s electricity data

HoustonKemp.com 3

2. The impediments to accessing consumer’s

electricity data

On 1 March 2016, consumers and their authorised third party, obtained a right to access their own metering

data. Both in the lead up to that right coming into effect, and since, several reviews have examined the need

to improve consumers’ access to consumption data. This section provides a brief description of this work,

which provides the context for our project.

2.1 The right to access electricity consumption data

On 1 March 2016, the Australian Energy Market Commission (AEMC) introduced a new rule to facilitate

access to electricity consumption data. The objective of the rule is to make it easier for customers to access

their electricity data, thereby allowing consumers to make better and more informed choices about energy

products and services as well as save electricity costs.

The rule:

allows customers and authorised third parties to obtain their electricity consumption data from distribution network service providers (DNSPs) and retailers; and

requires retailers and DNSPs to comply with minimum requirements related to the format, timeframes and reasonable charges when providing the data.

To facilitate the rule change, the Australian Energy Market Operator (AEMO) has developed metering data

provision procedures. This established the minimum requirements for the manner and form in which retailers

and DNSPs must provide metering data to a consumer, or their authorised representative. These

procedures, developed in consultation with industry, gave some flexibility to DNSPs and retailers about

elements of the process, which in turn has led to some inconsistency in application.

2.2 The need to do more to facilitate access to consumer data

Despite the introduction of a right to access electricity data, several additional impediments to accessing

consumers’ electricity data have been identified.

2.2.1 The Finkel review recommendations and government’s response

The Independent Review into the Future Security of the National Electricity Market (known as the Finkel

review) recognised that improving access to consumer energy data is key to consumer empowerment. In

particular, the Finkel review stated that: 1

Service providers, including a range of new technology providers, require increased access to

electricity consumption and other data, in order to assist consumers and find ways to help them

save money. Improved data access can also assist governments, policy makers and others to

better understand the market and the impacts of changes in the market.

Consumers should have access to their electricity consumption data in real time. They should also

have control over who, if anyone, can access that data….

…practical barriers limit consumers’ ability to easily access and use this information. For example,

they may face onerous requirements to verify their identity, may receive their data in a variety of

formats, and may receive their data up to 10 days after making their request. This is a long way

1 Commonwealth of Australia, Independent Review into the Future Security of the National Electricity Market: Blueprint for the Future, ,

June 2017, page 142-143


HoustonKemp.com 4

from an ideal in which data are available immediately in a form that suits the consumer, for

example to input into an app or website. Ultimately, energy retailers, third party energy service

providers and behind-the-meter automation software will play an important role in removing

complexity for consumers.

This led to the Finkel review recommending the following: 2

The COAG Energy Council should facilitate measures to remove complexities and improve

consumers’ access to, and rights to share, their energy data.

In response, the COAG Energy Council committed to support this recommendation in August 2017. This was

in line with previous commitments already agreed under the National Energy Productivity Plan, which

identified the need to have real-time access to their energy use profiles to facilitate the development of

innovative services and to inform consumers when making purchase decisions.

Retailers also made a related commitment during roundtable discussions with the Prime Minister, where

retailers agreed to provide feedback on progressing a range of matters including “implementation of right to

data and technologies for using QR codes and similar online comparisons.”

2.2.2 The Productivity Commission’s inquiry on data availability and use

The Productivity Commission (PC) recently completed an inquiry into data availability and use, including the

benefits and costs of improving the availability of data, and how it can be achieved in practice. The PC found

that allowing access to data would likely have large benefits in many sectors, including in the electricity

sector. That said, the PC also identified that there would be risks involved, and the risk would vary

depending on the nature of the data.

The Productivity Commission identified the need for significant changes to facilitate improved access to data.

Key recommendations include:3

A new Data Sharing and Release Act, and a National Data Custodian to guide and monitor new

access and use arrangements, including proactively managing risks and broader ethical

considerations around data use.

A new Comprehensive Right for consumers would give individuals and small/medium businesses

opportunities for active use of their own data and represent fundamental reform to Australia’s

competition policy in a digital world. This right would create for consumers:

powers comparable to those in the Privacy Act to view, request edits or corrections, and

be advised of the trade to third parties of consumer information held on them

a new right to have a machine-readable copy of their consumer data provided either to

them or directly to a nominated third party, such as a new service provider.

If implemented, the proposed comprehensive right for consumers to access their own data could mean the

data could be more than just meter data.

The Australian government has established a data availability and use taskforce to prepare a response to

the PC’s recommendation. In addition, the Australian government has announced its intention to introduce a

national consumer data right on a sector by sector basis, commencing with the energy, telecommunications

and banking sectors. Under the new consumer data right, consumers and authorised third parties would

have open access to their data in a standard, comparable, easy to read digital information. 4

2 Commonwealth of Australia, Independent Review into the Future Security of the National Electricity Market: Blueprint for the Future, ,

June 2017, page 143

3 Productivity Commission, Data Availability and Use Inquiry Report, 31 March 2017, page 2

4 Government’s response is available at https://ministers.pmc.gov.au/taylor/2017/australians-own-their-own-banking-energy-phone-and-internet-data, last accessed on 6 February 2018

https://ministers.pmc.gov.au/taylor/2017/australians-own-their-own-banking-energy-phone-and-internet-data

https://ministers.pmc.gov.au/taylor/2017/australians-own-their-own-banking-energy-phone-and-internet-data


HoustonKemp.com 5

2.2.3 Review into open banking in Australia

On 20 July 2017, The Hon Scott Morrison commissioned the open banking review to recommend the most

appropriate model for open banking in Australia. The purpose of the review was to examine how existing

arrangements could be altered to give consumers, and their authorised third parties, greater access to their

banking.

While the review focused on the banking sector, it also sought to consider how these recommendations

could be applied to other sectors, so that the high-level principles and framework are common across

different sectors.

Relevantly, the banking review also:

confirmed the development of a national Consumer Data Right (CDR), to be led by the Treasurer, to be the informed by the recommendations of the Open Banking Review;

that the CDR will be implemented economy-wide on a sector-by-sector basis, commencing with banking, energy, and telecommunications sectors; and

recommended a new regulatory framework under which an open banking regime would operate and the necessary instruments (such as legislation) required to support and enforce a regime’, including changes to the Privacy Act.

2.2.4 Several other reports have also highlighted the need to facilitate access to consumer data

In addition, there have been numerous other reviews that have recognised the importance of providing

consumer access to their electricity data, including:

the ACCC Retail Electricity Pricing Inquiry5, which has raised data access as one of the obstacles to good consumer outcomes, saying that “there are barriers to consumers accessing information about their electricity usage and using this information to compare and choose an offer.”; and

a joint report from the Climate Change Authority (CCA) and Australian Energy Market Commission (AEMC)6 on delivering affordable, secure and lower emissions generation also recognised that consumer electricity data would be important for developing demand management services.

2.3 Addressing the impediments

There have been a number of activities seeking to address the remaining impediments to accessing

consumers’ energy data.

2.3.1 Energy Consumers Australia data portability project

Energy Consumers Australia (ECA) has been working on a data portability project to help facilitate consumer

access to their own electricity data. ECA released a discussion paper, which proposes an industry led

process for facilitating access to electricity data that seeks to maximise the associated benefits for

consumers.

The objective of the data portability project is: 7

That Distribution Network Service Providers (DNSPs) will build a process whereby third parties,

acting on the explicit informed consent of customers, can be provided with NEM12 meter data that

they already hold through a real-time transaction through the B2B platform

5 Australian Competition and Consumer Commission, (2017), Retail electricity pricing inquiry: Preliminary report, 22 September.

6 Climate Change Authority, and Australian Energy Market Commission, (2017), Towards the next generation: Delivering affordable, secure and lower emissions power, June.

7 Energy Consumers Australia, Electricity Meter Data Portability Discussion Paper, July 2017, page 8. Available at energyconsumersaustralia.com.au/publication/electricity-meter-data-portability-discussion-paper


HoustonKemp.com 6

The ECA considers that there are two primary benefits from facilitating access to consumers’ electricity data,

namely:

1. Helping consumers better understand the likely bill impacts of alternative retail market offers; and

2. Maximising the value of consumer investments in distributed energy resources.

The ECA considers that network businesses would also benefit from 2), and so suggests that DNSPs are

best placed to develop the necessary processes.

2.3.2 AEMO B2B procedural development work stream

On 30 June 2016, the AEMC made a final rule determination that made amendments to the electricity

business to business (B2B) framework. The purpose of these amendments was to open up competition in

metering and related service, which were part of the Power of Choice market reforms.

The framework sets out a standard form for data communication between businesses for certain services

associated with small customer meters. This process was limited to communications and did not address

supporting business processes, such as processes for verification of consent. The final rule determination

also specified that AEMO upgrade its B2B e-hub, which can only be used by accredited parties.

AEMO worked with the Information Exchange Committee to implement updates to the B2B framework

needed to support the Power of Choice reforms, such as the introduction of new market participants,

including the metering coordinator. The AEMO has updated the B2B platform (the e-hub) to enable

registered participants to transact with each other more quickly than its previous iteration.

2.3.3 Victorian energy data hub

The Victorian Department of Environment, Land, Water and Planning (DELWP) is currently investigating the

potential to create a central, digital hub for the storage and access of electricity consumption data. The

consumer data hub would give consumers, and their authorised third parties, access to recent and historical

data on consumption, voltage, tariff and other electricity data.

2.4 Vision for future consumer energy data access

Given the changes occurring in the electricity sector, there is a need for consumers to have easy access to

their electricity data, to assist in evaluating the potential benefits of investments in new technologies (eg, the

size and type of solar PV or battery systems, or other energy efficiency technologies), alternative tariff

products and offers, or to directly manage electricity use to lower bills.

While consumers have a right to access consumption data, for this data to be both useful and accessible it is

important to also ensure that:

consumers have the information they need to make informed decisions;

consumer privacy is protected and any risks associated with the release of data are managed; and

data is provided in a format and via technology that delivers the information in an easy to use, cost effective and timely manner.

The first point recognises that fundamental shifts in the electricity sector means that consumers are

increasingly needing to make more complex decisions about electricity and its supply. The ‘right’ decision will

depend on the individual circumstances of the consumer, and so access to historical electricity use data will

be critical to them making informed decisions.

The second point relates to consumer’s right to privacy and the risks associated with data being potentially

fraudulently acquired by someone. Facilitating access to consumer data should also take account of a


HoustonKemp.com 7

consumer’s right to privacy. That is, there needs to be proper frameworks put in place to ensure that

information sharing is secure and there are safeguards to protect and prevent fraud.

The last point refers to the need to ensure that data is provided in a useable, machine readable format, and

that both the format and delivery technology are likely to change in the future. The scope and format of data

would also need to consider and facilitate the primary use cases for data, recognising that it the data is an

input into another application. Any access framework needs to be sufficiently flexible to accommodate future

changes in needs. Other relevant considerations are that any arrangement for access to data should be

practical, in that it should be implementable in the short term, and not impose unnecessary costs to the

parties involved.

In our opinion, the objective for a consumer electricity data access framework should be:

To facilitate on-demand access by retail customers or a customer’s authorised representative to


Facilitating access to consumer electricity data A framework for facilitating access to consumers’ energy data

HoustonKemp.com 8

3. A framework for facilitating access to

consumers’ energy data

In this section, we set out a high-level framework for facilitating access to consumers’ energy data that seeks

to provide:

verification and consent processes that are uniform, simple, and facilitate on-demand access to data by third parties while complying with privacy obligations;

data scope and formats that provide the necessary information in a data format that is consistent, with a framework to provide flexibility and manage necessary changes over time; and

an approach to data delivery that is uniform, simple and facilitates on-demand access to data by third parties.

We believe that a framework that delivers these outcomes will facilitate improved access to consumers’

energy data by third parties.

3.1 High-level framework to facilitate access to consumers’ energy data

In our discussions with stakeholders over the course of this project, it has become apparent that there are

several obstacles to achieving improved access to consumers’ energy data. These include:

different assessments by distributors and retailers as to what is required to manage privacy risks and obligations, particularly due to concerns about reputational risks;

absent clear funding arrangements, strong incentives to minimise the cost of providing data, which leads to differences in data formats, changes in data formats over time, and different delivery mechanisms which are not focussed on timely delivery; and

incentives to minimise data provided due to concerns, particularly by retailers, that the information may be used to increase a competitor’s market share.

Feedback from most stakeholders indicate that there is strong support for a government led solution

because:

government involvement could help mitigate potential damage to the brand in the event of a leak;

means third party service providers would have a ‘seat at the table’; and

less risk of delays as it does not require consensus or agreement amongst industry.

It follows that we do not believe that distributors or retailers are well placed to work collaboratively to develop

the necessary frameworks and systems to address the impediments to accessing consumers’ data that have

been identified.

We are therefore proposing that a consumer electricity data access scheme be developed by the

Australian Energy Market Operator (AEMO), to facilitate on-demand access by consumers or an

authorised third-party, to consumers’ energy data.

We believe that AEMO is an ideal organisation to develop and administer the scheme given their knowledge

of the sector, and their current role of registering market participants as part of the B2B e-hub.

Implementation of the proposed consumer data access scheme would require changes to the National

Electricity Rules (NER). It follows that we propose the NERs be changed to:


HoustonKemp.com 9

require AEMO to develop a consumer data access scheme within 12 months of the rule coming into effect;

set out the objective of the scheme to guide its development; and

set out the elements to be included in the scheme.

Based on our discussions with stakeholders, we believe that the objective of the proposed electricity

consumer data access scheme should be:

To facilitate on-demand access to retail customers or a customer’s authorised representative to


The key elements of the objective are:

the facilitation of on-demand access, to ensure that third party data providers are able to immediately respond to consumer’s requests for their services. This acknowledges that many data related services are likely to be most valued by consumers at the point when they are making a decision or investigating options about electricity product offerings;

by retail customers or a customer’s authorised representative, to acknowledge the importance of a representative being authorised by the consumer to provide advice, given our focus on access by third party data providers, although the framework should also apply to a customer seeking access to their own data; and

to consumer electricity data, which is intended to encompass both the customer’s current right of access to data that is collected via a customer’s meter (ie, metering data), and other consumer electricity data that might be needed to facilitate the provision of innovative value-added services to consumers.

While we expect that the initial focus of the scheme will be on metering data, we believe there may be merit

in considering extending the data to which the proposed consumers data access scheme applies to. This is

discussed in further detail in section 5.

In our opinion, the scheme should provide details on:

the process and requirements for accreditation of customer authorised representatives, including information requirements and timeframes for accreditation;

provide a uniform approach to retail customer verification and consent procedures;

provide a centralised, streamlined and useable/consumer-friendly approach to the provision of metering data to customer authorised representatives;

specify a uniform format in which retail customers’ metering data must be provided to customer authorised representatives;

specify the procedures for requesting and approving changes to the detailed data format by any party; and

be developed within 12 months of the rule coming into effect.

The proposed consumer electricity data access scheme would provide the detailed elements of the

framework for facilitating access to consumers’ energy data. While the specific details as to the method and

form of access is proposed to be the subject of development by AEMO, in the remainder of this report we set

out an approach to the key elements of the scheme that we believe would be sufficient to facilitate greater

access by third parties to consumers’ electricity data.

3.2 Implementation of a consumer data access scheme

To implement the proposed consumer data access scheme, we believe there is a need to ensure that the

scheme satisfies privacy obligations as set out in the Privacy Act 1988 (Commonwealth) for residential

customers, the national electricity rules, and the common law for business customers.


HoustonKemp.com 10

In our opinion, this could be achieved by:

the scheme being certified by an appropriate authority for privacy protection (eg, the Office of the Australian Information Commissioner or the Australian Competition and Consumer Commission (ACCC)); and

upon certification, distributors and retailers being deemed to comply with their privacy obligations so long as they comply with the requirements of the certified consumer data access scheme.

That said, the recently released Review into Open Banking report8 has proposed developing a common

legislative and rules framework to facilitate access to consumer data across any sector, which would give the

ACCC a role in accrediting third party’s seeking access to consumer data. This would be analogous to the

process we are proposing for the energy sector, although these responsibilities are proposed to be given to

AEMO.

Depending on the acceptance of the open banking framework, it might be possible for AEMO to develop its

preferred accreditation framework for the electricity sector, with the ACCC certifying it to be consistent with

the legislative and rules framework developed to be consistent across all sectors. This approach would strike

a balance between the desirability of continuing progress towards facilitating access to consumers’ electricity

data whilst being consistent with proposals being developed for other sectors.

We expect that our proposed approach to developing a consumers’ electricity data access scheme will

require legislative changes to provide assurance to distributors and retailers that their data privacy

obligations are fulfilled through compliance with a uniform consumer data access scheme. Absent a

certification process, the privacy obligations are expected to reside with the holders of consumer energy

data. Any differences in opinion as to how those obligations should be fulfilled while providing data to

consumer authorised representatives will need to be resolved by the AEMO in the development of the

consumer data access scheme.

3.3 Questions for stakeholders

1. Is the proposed objective for the consumer electricity data access scheme appropriate?

2. Should AEMO or an alternative agency be given responsibility for developing the consumer electricity data access scheme?

3. Are there additional elements that the scheme should incorporate to facilitate access to consumers’ electricity data by authorised representatives?

4. What changes can be done in the short term without a rule change and what changes require a rule change to implement?

8 Review into Open Banking: giving customers choice, convenience and confidence, December, 2017.

Facilitating access to consumer electricity data Verification of authorisation procedures

HoustonKemp.com 11

4. Verification of authorisation procedures

A key element of the proposed consumer data access scheme will be the uniform procedures for the

verification of authorisation of a customer representative to access a consumer’s electricity data. These

procedures need to balance privacy concerns and the risk of fraudulent use of consumers’ energy data, with

the need to provide on-demand access to the data by consumer authorised representatives.

This section sets out our thoughts as to procedures that we believe would strike an appropriate balance

between privacy risks and facilitating access to data. We commence by briefly discussing the current

arrangements for obtaining verification of customer consent, before describing the privacy obligations, and a

proposed procedure for verification of authorisation.

4.1 The current approach to verification of authorisation

The current approach to verification of authorisation places the obligation on retailers and DNSPs to fulfil

their Privacy Act obligations. AEMO’s metering data provision procedures require that retailers and DNSPs

publish information on its process for verification and consent. In addition, the procedures stipulate the

timeframe within which data must be provided when requested, being:

three business days for a data request relating to a single retail customer;

six business days for data requests related to two to 100 retail customers; and

an agreed timeframe when the request relates to accessing consumption data for more than 100 retail customers.

These timeframes are in addition to the timeframe for file transfer discussed in section 5 below. The timeframes are currently sufficiently long that retailers and DNSPs are able to engage in relatively lengthy processes to verify first the identity of the authorising party, and that authorisation of the third-party has been given. However, these timeframes can result in significant wait times of up to 13 business days for a single data request.

These processes can differ substantially across DNSPs and retailers. However, in general a third-party seeking access to a consumers’ electricity data is required to provide some or all of:

the customer’s details, such as name, date of birth etc;

the account details, such as account number, physical address of the property, and the national meter identifier (NMI);

contact details of the primary account holder, such as an email or phone number; and

the period of time over which data is being sought.

The retailer or DNSP uses this information to ensure that the customer and account relates to the property

for which data is being sought, and for the period of time over which the data is being sought.9 Once the

identity of the account holder has been verified for a particularly NMI, the second stage is to verify that the

account holder has provided consent to the third-party representative.

This typically requires the provision of additional information, which can include one or more of:

a signed consent form or some other written proof of authorisation;

information on the third-party company acting as a representative of the consumer; and

9 We understand that it is currently difficult for a distributor to determine the time period over which a current customer can legitimately

authorise data access for a given NMI. This is because distributors typically do not keep records of when a customer moved into a premise, or become the account holder for a particular NMI.


HoustonKemp.com 12

a contact person at the company.

We understand that DNSPs and retailers typically require the signed consent form to be provided by email.

The current approach to verification is inconsistent with our proposed requirements for a consumer data

access scheme because, by requiring proof of authorisation at the time of a transaction and the need to

verify this proof, it is not possible for on-demand data to be provided. It follows that changes would need to

be made to satisfy the objective for the proposed consumer electricity data access scheme.

4.2 What do retailers and distributors need to do to comply with the

requirements of the Privacy Act?

The Privacy Act places obligations on retailers and DNSPs on how they handle personal information. The

challenge for retailers and distributors is how to balance these privacy obligations against the requirement to

provide access to metering data by consumer authorised representatives.

4.2.1 Requirements under the Privacy Act

The Privacy Act regulates how personal information is handled. It applies to the private sector and not-for-

profit organisations that have an annual turnover of $3 million and above, and most Commonwealth

government agencies.

The Privacy Act defines personal information as:10

information or an opinion about an identified individual, or an individual who is reasonably

identifiable:

a) whether the information or opinion is true or not; and

b) whether the information or opinion is recorded in a material form or not.

Examples of personal information include an individual’s name, signature, address, telephone number, date

of birth, medical records, bank account details and commentary or opinion about a person. It follows that a

consumer’s identifiable electricity consumption data is likely to be considered as personal information as

defined by the Privacy Act.

The Privacy Act places obligations on collectors of data to ensure that personal information held is managed

in a manner that is consistent with Australian Privacy Principles (APPs). The APPs require that an entity

holding personal information takes reasonable steps to protect the information from unauthorised access or

disclosure.

The APPs are not prescriptive and require businesses to examine how the principles apply in their situation.

The obligations are open to interpretation by organisations that collect and manage data.

4.2.2 Requirements for business customers

The Privacy Act may not apply for business customers as the data may not be classified as personal

information. That said, there are other laws that apply, namely the common law. As the open banking inquiry

points out: 11

The common law imposes a contractual duty of confidentiality on banks not to disclose the affairs

of their customers — whether individuals or businesses — unless the disclosure falls within four

limited exceptions. In addition, the law of equity can impose an obligation on banks to maintain

confidence. Further, an obligation to treat customer’s information confidentially is often included

as a condition in the contract a customer has with their bank for services. These obligations of

10 Section 6, Privacy Act 1988 (Cth).

11 The Australian Government the Treasury (2017) Review into Open Banking.


HoustonKemp.com 13

confidence extend to all types of customers and therefore give business customers a claim against

their bank if their information is not kept secure.

In addition, the NERs also impose confidentiality obligations on registered market participant, namely:12

Each Registered Participant must use all reasonable endeavours to keep confidential any

confidential information that comes into the possession or control of the Registered Participant or

of which the Registered Participant becomes aware.

An exception to both the NERs and the common law is when a consumer has provided consent for their

information to be provided to a third party.

4.2.3 Consequences resulting from the provision of electricity data to a third party

A key consideration when determining the reasonable steps to protect personal information is the

consequences that might result from a breach. The Productivity Commission has recently identified six key

consequences of a breach of privacy to consumers, namely:13

scope for discrimination based on the data provided;

a loss of control over the boundaries around the ‘you’ that the world sees;

potential for reputational damage or embarrassment;

identify fraud;

other criminal misuse of the data; and

commercial harm.

The risk associated with electricity will vary depending on the nature of the data and the consumer. The

particular risk for residential users is the concern about loss of control of boundaries whereas for commercial

businesses, there could be commercial harm, eg, electricity data could be used to infer business activities.

There is also a risk that meter data could be used to assess patterns of when someone is at home or not,

particularly for ‘real time’ data on an ongoing basis. In contrast, the risk of identify fraud or commercial harm

seems to be relatively low.

4.2.4 Consequences for data providers

There are two main consequences for data providers arising from the provision of data to an unauthorised

third party, namely:

the legal consequences from breaching the Privacy Act, the NERs or common law; and

potential damage to a company’s reputation.

Breaches of the Privacy Act, the NERs or common law can lead to the imposition of a fine, depending on the

severity of the breach and the actions taken by the data provider to manage the risks.

The second consequence is the potential damage a data provider’s reputation if there is a large-scale data

breach, or if the public perceives that a data provider did not have sufficient procedures in place to protect

customer data. Where the data provider is operating in a competitive market, this can lead to customers

switching to an alternative competitive service provider. For electricity network businesses, while switching

might not be a direct risk, the impact on confidence in the service provider can affect its legitimacy to operate

its service.

12 NERs 8.6.1(a). This applies to both residential and business customers.

13 Productivity Commission, Data Availability and Use Inquiry Report, 31 March 2017, page 8-9.


HoustonKemp.com 14

4.3 A proposed approach to verification – an accreditation scheme for

third parties

To address the need to balance privacy obligations with facilitating third parties to access data on-demand

and in a uniform manner, we propose that the electricity sector adopt a process of pre-accreditation of third

parties seeking access to consumers’ data. This process would allow privacy obligations to be managed

through an agreement to systems and processes, ahead of a data provider seeking access to the specific

data of a consumer.

In this section we briefly explain how our proposed accreditation scheme would operate.

4.3.1 Objective for an accreditation scheme

As we have outlined earlier, current verification processes create an impediment to third party access to

consumers’ data because:

each distributor/retailer has a different process, reflecting different approach to managing privacy risk, which requires third parties to satisfy different processes and obligations;

there is typically an obligation on third parties to provide verification/consent at the time of each data request, which can lead to delays in the provision of data; and

the method of proving the receipt of a consumers’ consent is cumbersome, typically relying on signed forms as proof of authority, further contributing to delay in the provision of data.

In our opinion, the last impediment is of particular concern because it limits the scope for verification and

consent processes to be automated, and it further represents a significant impost on consumers. Further,

given that distributors and retailers typically do not collect a sample signature from their customers, it is

doubtful that the current process provides any real protection against fraudulent access to a consumers’

electricity data.

To provide on-demand access to consumers’ energy data, while managing privacy obligations, we believe

that the framework for accreditation should satisfy the following objectives:

comply with the requirements and obligations as set out in the Privacy Act, the NERs or Common Law;

allow an accredited third party to access data from any retailer or distributor; and

provide a standardised process for proving identity and authorisation.

4.3.2 Overview of an accreditation scheme

In the absence of easy electronic verification of customer identity and authorisation at the point of a data

request (say, via customer identity management through a retailer portal), we believe there is scope for an

accredited third-party data seeker to obtain and retain identity and authorisation information, for possible

subsequent audit by data providers. In practical terms this would mean that:

a third-party data seeker would seek accreditation prior to requesting data, which would require it to agree to (amongst other things) a process for collecting and retaining customer identity and authorisation information prior to seeking to obtain a consumer’s data from a distributor or retailer;

a third-party data seeker would provide a distributor or retailer with basic information about a customer that it is seeking access to data (eg, customer name, account number, NMI, dwelling address, etc) along with fulfilling its consent information requirements, consistent with the accreditation obligations;

a distributor or retailer would presume that an accredited third-party has obtained the necessary consents when receiving a request for access to a consumers’ data; and

audit processes of a random sample of requests would be used to ensure that fraudulent access to consumer’s energy data had not been obtained.


HoustonKemp.com 15

We understand that this is similar to existing processes used by retailers when sharing customers’ energy

data upon a customer seeking to transfer between retailers.

Process for accrediting third-party data seekers

The purpose of the accreditation process is to act as a first layer of protection from the fraudulent access of

consumers’ data by a third-party. To obtain accreditation, a third party would need to provide information

such as:

basic company information and key personnel, eg, ABN, address and phone number; and

the services to be provided by the company and the associated capabilities to provide the service.

The purpose of the assessment is to ensure that only fit-and-proper companies who are capable of providing

services can access the relevant data in an on-demand manner. Accreditation would also place privacy and

other information management obligations on the accredited third party, thereby providing consumers with

protections against fraudulent access and use of consumers’ metering data.

The application for accreditation would be assessed by the AEMO, consistent with the requirements of the

consumer data access scheme.

We are proposing that AEMO have responsibility for both developing the accreditation scheme and

subsequent implementation of the scheme by assessing parties seeking to become accredited. This reflects

our opinion that AEMO has the necessary governance arrangements and experience in accrediting market

participants. In addition, it would be well placed to monitor and enforce obligations imposed on third parties,

given it already provides similar roles in its market operations and settlement roles. This additional

responsibility would therefore be a natural extension of its existing functions.

Information to be provided by a data access seeker to a distributor or retailer when seeking

consumers’ data

To gain access to a consumer’s electricity data, a third party would need to provide additional information,

such as the customer’s name, address and NMI. At a minimum, the third party would need to provide

sufficient information for the data provider to identify the relevant meter data to provide, eg, NMI or address.

Additional information could be provided when an authorised third party asks for data to check if consent has

been provided. The additional information would need to be machine-readable to enable the provision of

data on an on-demand basis. One possibility is to ask third parties to provide information that is currently

used to verify identify when a first party is asking for the data, eg, account number, customer name, address,

and phone number etc. However, anecdotal information suggests that retailers and DNSPs currently have

difficulty matching the information on record and provided by the customer, eg, address could be written in a

different format (st vs street) and a customer could provide a different number than that provided on their

official record.

An alternative would be to ask third parties to provide information that is easier to match, eg, postcode, first

name on the account or account number. However, the key question is what would represent reasonable

steps from a legal perspective and a useability perspective.

Presumption that an accredited third-party has fulfilled its consent obligations

Once identified, a distributor or retailer to whom a data request is being made would presume that the

consumer has provided consent to the third-party to release the information. This aspect of the accreditation

scheme is critical to the delivery of on-demand metering data to third parties. It relies on distributors and

retailers having confidence in the accreditation process and associated compliance auditing processes, in

terms of it satisfying their privacy obligations.


HoustonKemp.com 16

Process for auditing that proper consumer consent has been given

The proposed process for accreditation and management of consumer consent relies on an ex-post audit

process with associated penalties for non-compliance, instead of requiring the verification of identity and

consent at the time of each data request.

It follows that the accreditation scheme would need to set out both the acceptable methods for obtaining and

recording consent, and place obligations regarding record keeping for the consent. There are numerous

possible methods that could be used, including, but not limited to:

a third party having evidence to support that it had adequately notified the consumer of its intention to collect their meter data, and the consumer having given consent by recorded verbal means or digitally by checking the agreement button; or

sending email correspondence to the consumer and recording a return email acknowledging consent.

The process will also need to explicitly inform consumers on whether they have consented to providing one-

off access or access on an on-going basis.

In seeking accreditation, a third-party could accept a particular method that had been previously recognised

by AEMO, or propose an alternative method for obtaining consent to be approved by AEMO.

In the event where a violation has occurred, then there would be penalties, such as a warning of non-

compliance, financial penalty, or temporary or permanent loss of accreditation.


5. Are there alternative approaches to managing verification of consumer identity and third-party

authorisation that should be considered and which are consistent with the scheme objective of providing

on-demand access to data by authorised third parties?

6. Should AEMO or another agency be given responsibility for accrediting third parties?

Facilitating access to consumer electricity data What data should be made available?

HoustonKemp.com 17

5. What data should be made available?

We are proposing to provide access via the consumer data access scheme to consumers’ metering data,

which is consistent with the pre-existing right provided in the NER. In this section we discuss the types of

electricity data that are currently collected, our reasons for the data scope that we are proposing initially, and

opportunities in the future to provide access to additional data.

5.1 The current data right and availability of consumers’ data

5.1.1 Consumer’s right to access data

Under the NERs14 and the National Energy Retail Rules15, consumers and their authorised representative

are provided with a right to access to metering data from either distributors or retailers. AEMO, through the

metering data provision procedures, specifies the minimum manner and form in which the retail customer’s

metering data must be provided upon request, including:

a detailed data format and summary data format for interval metering data; and

a summary data format for accumulated metering data.

The summary data provides a high-level overview of electricity consumption for the consumer in the past two

years. The summary data is required to be provided in PDF format, and its primary purpose is to help small

retail consumers understand their electricity consumption.

The detailed data represents the raw data that underpins the summary data, eg, 30-minute electricity

consumption data. Retailers and DNSPs are required to provide this data in a CSV format and meet the

minimum requirements in the Meter Data File Format specified by AEMO. However, as the format is a

minimum specification, there is significant discretion provided to retailers and distributors as to the format of

the data provided, which can lead to inconsistencies in data supplied.

5.1.2 Data that is currently collected and so potentially available for access by consumers

Consumer related electricity data can be grouped into four main categories, comprising of:

customer related data, such as name of account holder, contact details and time period of account;

standing/connection point data, such as the NMI, supply address, meter type, and details on solar panel and battery installation;

meter data, such as consumption by time interval, voltage and frequency; and

pricing data, such as retail and network tariff, total bill, and other fees and charges.

Customer related data is collected by retailers when a consumer first signs up with a retailer. Retailers share

this information with DNSPs in case DNSPs need to make emergency contact with the account holder. We

understand that DNSPs record information on the current account holders but do not necessarily hold

information on previous account holders or when the account was created.

Connection point data is information related to the connection point and is relatively static over time.

Relevant information includes the supply address, meter types installed, and capacity of solar panels and

batteries where relevant. Meter related data, including supply address, meter type and time installed, is

collected by metering providers, and shared with other market participants via MSATS procedures.

14 NERs, 7.15.5(d).

15 NERRs, 56A(1).


HoustonKemp.com 18

The Clean Energy Regulator collects basic information on solar panel installations. Currently, no one is

responsible for collecting information on battery installations but procedures to share this information are

currently being developed.

Meter data providers are responsible for collecting meter data following the AEMO meter procedures. Meter

data is then shared with other market participants via AEMO’s B2B information hub and is used for market

settlement. However, information on frequency or voltage is not collected as part of this process as this is

useful mostly for network management purposes.

Retailers are responsible for calculating the tariffs and charges that are applicable for a consumer. Retailers

publish the tariffs they offer to consumers on their website, and current tariff offers are also made available

via tariff comparison websites.

The offer that a retailer makes to an individual consumer will vary depending on the jurisdiction and network

area of the address, as well as the meter type installed at the house. Retailers also commonly provide

consumers with discounts for on-time payment and for the bundling of gas and electricity supply. Because of

these billing complexities, we understand that there are around 4,000 to 5,000 generally available market

offers that are currently available.

The actual retail tariff offer that a consumer is currently assigned to may differ from offers that are generally

available at a given time. This reflects changing product offerings, and so a consumer may be on a legacy

tariff. It follows that the actual number of tariffs that exist could be significantly more than the number of

generally available tariffs. Retailers are obliged to disclosure relevant charges on a consumer’s electricity bill

but this information is not shared with other market participants.

5.1.3 Proposed future changes to data being collected and shared

AEMO has proposed a number of changes to the information that is shared amongst registered market

participants to accommodate a number of changes that are occurring in the market, eg, the introduction of

five-minute settlement. The proposed changes will mean that information shared via AEMO’s B2B

information hub is expected to:

change from 30-minute intervals to five-minute interval data;

change from net meter data (ie, the sum of electricity imports and exports to the network) to gross meter data (ie, the availability of both total imports and total exports to the network); and

increase coverage of meters to include data collected from accumulation meters.

In short, these proposed changes mean that the availability of consumers’ metering data available to be

shared via the AEMO B2B information hub may increase substantially over the near term, by covering more

meters and becoming more detailed.

The AEMO B2B hub is used to transfer information for the purposes of market settlement. Given this, we

expect that the scope of information shared via the hub will align with any future technology or regulatory

changes that could have significant implications for market settlement, such as changes to tariff structures

such as greater use of demand based tariffs.

5.2 Assessing the merits of facilitating access to additional consumer data

As a minimum, consumers should have access to metering data that is readily collected and available via the

AEMO B2B data hub. This provides the basis for updating and expanding on the data available as the data

available on the hub changes over time.

The additional data that are possible candidates for consumer access are:

standing/connection point data, ie, data about the meter type, supply address etc;


HoustonKemp.com 19

customer related data, ie, the name of the account holder, time period of the account and assigned tariff product;

pricing data, ie the retail and network tariffs, total bill and other fees and charges including discounts, as applicable; and

estimates of meter data – to provide de-identified information to a new resident.

We examine in the remainder of this section the reasons why some of this data might be made available in

the future.

5.2.1 Possible uses for consumer electricity data

Our discussions with stakeholders indicate that the main reason for authorised third parties to access

consumers’ electricity data is to develop tailored, value added services to individual consumers, eg,

informing consumers what the best retail tariff is or to estimate the potential financial benefits from installing

solar PV panels or storage.

The key electricity data needed to support these use cases are both a consumer’s specific historic

consumption data as collected via the consumer’s meter, information on the tariff product that the consumer

is currently assigned to, and the pricing data associated with the tariff.

Relevantly, the current right of consumers to access consumer electricity data relates to metering data but

not tariff or pricing information. That said, existing rights could be extended if the PC’s recommendation to

introduce a new consumer right to access their data is implemented.

Feedback from stakeholders suggests that there are mixed views about the inclusion of tariff product and

pricing data. Some stakeholders suggest that this information help third parties improve the accuracy of their

estimates, and so help better inform customers, but this would only provide limited benefits.

However, many stakeholders did not support the inclusion of tariff product and pricing data. For example,

Energy Consumers Australia believes that the data scope should be limited to meter data, and so other data

should be excluded. Several stakeholders also indicated that this could be costly and difficult to implement,

given the large variation in product details and price (eg, legacy contracts, staff discounts, on time payment

and bundling discounts etc). Retailers were also concerned that this information could be used by their

competitors as a market research tool.

With the emergence of new technologies, such as batteries or electric vehicles, and possible regulatory

changes, such as the introduction of cost reflective tariffs, we expect that future use cases could require very

different data than existing use cases. Consequently, the data scope, and possibility the corresponding

verification and consent process, would likely need to change based on the changing needs of consumers.

5.2.2 Cost and implications of providing different types of data

The cost of sharing the data will depend on the nature of the data:

data that is already collected and shared among market participants, eg, meter and MSATS data: and

data that is collected by an organisation but not shared with other market participants, eg, retail tariffs and other meter data such as frequency and voltage.

Data that is currently shared amongst market participants would follow standard definitions and sharing

procedures. Given this, the data should be relatively low cost and easy to share.

Data that is collected but not currently shared among market participants would likely involve higher cost to

provide to third party participants. This is because standard definitions would need to be developed so that

the information is machine readable and firms may not have the necessarily infrastructure to share this

information on an on-demand basis.


HoustonKemp.com 20

5.2.3 Consistency in data format and definition

A key issue with the current arrangements is that inconsistent data formats by a single organisation over

time are hindering the ability for third parties to provide their services in an effective and efficiency manner.

Feedback from stakeholders suggest that the key requirements needed is that the data is provided in a

machine-readable form, and that inconsistencies between organisations was not a large issue. In other

words, mapping could be used to convert different formats into a format required by third parties without

significant burden, so long as organisations each provided a consistent format through time. We expect that

both delivery options discussed in section 6 would deliver the required outcomes to meet the needs of third

party service providers.

5.3 Recommended scope of data to be provided to third parties

Consistent with the current right of access to consumers’ electricity metering data, we believe that there is

currently no need to provide additional data (eg, pricing or tariff product information) as part of the proposed

consumer data access scheme. This is because feedback from stakeholders suggests that tariff product and

pricing information would be costly to provide, and is of limited value to existing third-party use cases. It is

also something that users can provide by giving third parties a copy of their bill. That said, we believe that

the value of this information could change in the future as new technology and cost reflective tariffs are

introduced.

Of the metering data that is collected, we believe that the proposed consumer data access scheme should

seek to minimise the cost of collecting and sharing data given currently anticipated use cases. Absent a clear

need to access voltage or other metering data, this would focus the data scope on data that is already

shared amongst market participants, ie, meter consumption data as collected and shared for market

settlement purposes.

It follows that pricing or tariff assignment data should not be included in the scope of data provided in the

immediate future, because consumers have ready access to this information via their bill, which can then be

shared directly with a third party as needed. An important reason for not including pricing or tariff assignment

information is the anticipated costs that would likely be incurred by retailers/distributors in providing this

information as part of a data request, given the complexities surrounding retail tariff structures and products.

Further, it is not clear that there is currently a clear use case for sharing other meter data. Given this information is not readily shared between market participants making it available would likely lead to the incurrence of costs with little benefits for consumers.

Finally, our proposed data scope for the proposed consumer data access scheme reflects anticipated needs for access to consumer data to support value added services. We expect that new business models and opportunities may arise over time, which will likely require the data scope to be revisited, eg given the change to five-minute market settlement. It follows that there is merit in the consumer data access scheme having a process by which consumers or third parties can propose the provision of additional consumer data as part of the scheme. AEMO could then assess the anticipated costs and benefits, and make a decision as to whether such data should be provided taking into account stakeholder submissions.


7. Should authorised and accredited third parties be given access to more than just a consumer’s metering

data upon the commencement of the data access scheme?

8. What are the arguments for and against providing third party access to retail and/or network tariff data?

9. What changes are required to existing AEMO metering data formats to facilitate access by third parties

to consumer electricity data?

Facilitating access to consumer electricity data Handling requests and data transfer

HoustonKemp.com 21

6. Handling requests and data transfer

The final element of a consumer data access scheme that we have considered relates to the approach to

handling requests for consumers’ metering data and the digital transfer of the data to the requesting party –

to either the consumer or authorised representative.

In this section, we first describe current arrangements, before setting out our evaluation of a preferred

approach.

6.1 Current process for providing data

The current process for transferring consumers’ electricity consumption data to third parties is managed by

each distributor and retailer, in compliance with its obligations to provide metering data upon request.

Individual requests from an authorised representative are submitted to the relevant retailer/DNSP, and data

is then transferred directly to the representative. Retailers and DNSPs are required to meet the following

timeframes for the transfer of data, namely:

10 days for a single request;

20 days for a request involving two to 100 retailer customers; and

by mutual agreement for a request involve more than 100 retailer customers.

The specific process and requirements for requesting data differs for each retailer and DNSP. In general,

most DNSPs and retailers require an authorised third party to provide a signed consent form and information

on the customer, provided via an online form or email.

We understand that most DNSPs and retailers have a largely manual process to deal with third party

requests, eg manual checking of the consent form. While these processes can meet the required timeframes

specified above, they are not designed to provide data on-demand, particularly for bulk requests.

DNSPs and retailers currently only receive a small number of requests from authorised third parties. We

understand that only 0.5 per cent of consumers have requested data via an authorised third party, with those

third parties being mostly solar panel providers. As a rough estimate, this equates to around 40,000 requests

for consumer data by third parties each year. Feedback from stakeholders suggest that the cumbersome and

time consumer nature of the existing processes is one of the reasons for the low number of current requests.

The low number of requests means that it can be difficult to justify investing significant amounts into

upgrading the system. We estimate that the current cost of managing transactions is around $170 to $340

thousand per year for all retailers and DNSPs16. It follows that the current approach is likely to be a cost-

efficient way for retailers and DNSPs to meet their current obligations under the NERs. That said, existing

processes are principally manual and not easily scalable, and so may not be viable if the volume of requests

increase significantly.

Figure 1 provides a diagrammatic representation of the current digital flow of data from a household to

DNSPs and retailers, and then onto a requesting authorised third party, under existing arrangements. The

key feature of this system is that all distributors and retailers currently receive metering data from metering

providers, via AEMO’s e-hub. However, an authorised third party seeking multiple consumers’ metering data

must interact with the systems and processes of each relevant distributor or retailer, to access the requested

data.

16 This assumes that it takes DNSPs and retailers around 5 to 10 minutes to deal with a data request, and the staff cost plus associated

overheads is $50 per hour. Our rough estimate does not take into account any system or process costs that might have been incurred to establish the associated systems, or the costs of managing enquiries from third parties seeking access to data.


HoustonKemp.com 22

Figure 1: Overview of existing data flows

6.2 Options for handling requests and data transfers

We expect that to satisfy the requirements for on-demand provision of consumers metering and data,

modifications would need to be made to the current approach to the provision of data to requesting

authorised third parties. This is to allow for the on-demand matching of requested consumer’s identity and

the subsequent provision of the relevant and available data.

In addition, the process of data transfer should be sufficiently flexible to accommodate the likely need for

changes over time, as changes in both digital technology and data scope arise.

In this section we examine the relative merits of creating a centralised data repository managed by AEMO,

as compared to maintaining a decentralised approach where distributors are principally responsible for data

provision.

6.2.1 A centralised data provision approach

The first option involves establishing a centralised data repository to handle requests for data from

authorised third parties. The central database would store consumers’ metering data and associated identity

information (eg, NMI and address), that is currently transferred between market participants, eg, data shared

via AEMO’s e-hub and MSATS, and consumer information currently shared between retailers and DNSPs.

Under this approach, an authorised and accredited third party would access data via an API that is linked to

the centralised data repository and maintained by AEMO or a private data repository provider.

A centralised data repository would provide a link between the NMI, the account holder, and metering data

thereby creating a complete record of metering data for all residents within a dwelling over time. One benefit

of this approach is that it would make it possible for a consumer to readily access metering data from a

number of premises that a consumer was the account holder of, over a given period of time.


HoustonKemp.com 23

From a practical perspective, this approach would also mean that retailers and DNSPs would not need to

incur additional direct costs to develop duplicate systems and processes to satisfy the on-demand data

provision objective. Instead, any AEMO costs associated with creating the system would be done once and

recovered via market fees.

Two potential options for implementing a centralised data provision approach include:

the creation of a separate central database, which would store all the relevant information required; and

allowing authorised third parties to have direct access to the AEMO e-hub.

Figure 2 provides a diagrammatic representation of data flows across a centralised data provision approach.

Figure 2: Overview of a centralised data provision approach

We would expect that the centralised data repository could be implemented as an incremental addition to

AEMO’s existing information exchange systems for both implementation options. This would allow the

transfer platform to benefit from existing data transfer systems, and thereby avoid the need to duplicate

existing systems. The incremental costs would therefore involve:

project management and consultation with stakeholders;

implementing and maintaining a process for data transfer;

costs of setting up and maintaining the data repository;17

costs of developing and maintaining a data portal and associated API; and

system testing prior to rollout.

17 This could be avoided if third parties have direct access to the AEMO e-hub. However, this would assume AEMO’s e-hub’s

capabilities would be expanded to include storage of historical information.


HoustonKemp.com 24

In total, we estimate that the centralised database option would cost around $0.5 – $1.5 million to establish, and have an on-going cost of approximately $0.1 – 0.4 million per year – Table 1. This represents costs of around $1.4 to $5.1 million over the next 20 years in present value terms18. These estimates reflect our understanding of similar systems given anticipated data storage requirements, and the costs of creating and maintaining API access to such a centralised database.

Table 1: Assumed establishment and ongoing cost of a centralised data provision approach

Cost category Establishment costs Ongoing costs per year 19

Project management $117,000-$468,00020 $11,700 – $93,600

Consultation $58,500-$117,00021 $5,850 – $23,400

Verification and data transfer system $78,000 – $312,00022 $7,800 – $62,400

Data warehouse $60,000 – $120,00023 $60,000 – $120,000

Data portal $50,000 – $100,00024 $5,000 – $20,000

Application programming interface $10,000 – $50,00025 $1,000 – $10,000

System testing $78,000 – $312,00026 $7,800 – $62,400

Total $451,500 - $1,479,000 $99,150 – $391,800

The large range in the estimated costs reflect the significant uncertainty regarding what the end

requirements of the system will be, possible integration costs, and the required capabilities and features.

Importantly, we have made the following assumptions:

the B2B e-hub can be readily integrated with the centralised database, and so there is no need to modify the B2B e-hub;

data procedures and definitions would be maintained as part of other processes, such as NEM12 file updates;

18 This assumes that implementation occurs in 2019 and system ‘go lives’ in 2020. The discount rate used is seven per cent.

19 Ongoing costs assumed 10 to 20 per cent of establishment costs, except for data warehousing costs.

20 The low case assumes it would take an ICT project manager (with a salary $180,000 and associated overhead costs 30 per cent) 6 months. The high case assumes that it would take an external consultant (at twice the cost of an internal staff) 12 months.

21 The low case assumes it would take an ICT project manager (with a salary $180,000 and associated overhead costs 30 per cent) 3 months. The high case assumes that it would take an external consultant (at twice the cost of an internal staff) 3 months.

22 The low case assumes it would take an internal business analyst (with a salary $120,000 and associated overhead costs 30 per cent) 6 months to set out the business requirements. The high case assumes that it would take an external consultant (at twice the cost of an internal staff) 12 months.

23 Low option based on Amazon RDS for MySQL Pricing, assuming a need for 10TB data. The high case assumes that the costs would be twice as much as the low option.

24 We have assumed that data is stored in requested in in the same format as those currently transferred amongst various parties. In other words, there is limited need to transform or manipulate the data. Low case assumes that a minimum viable product could be built for $50,000. The high case assumes that the costs would be twice as much as the low option.

25 Figures quoted in PC report suggests that initial cost of establishing an API would be around $10- 50 000, with most being about $20-25 000. Productivity Commission Data availability and use report, page 565

26 The low case assumes it would take an IT staff (with a salary $120,000 and associated overhead costs 30 per cent) 6 months to test the system. The high case assumes that it would take an external consultant (at twice the cost of an internal staff) 12 months.


HoustonKemp.com 25

the data would be stored using cloud storage, with appropriate data security features;

there is scope to leverage existing systems, and so no need for additional hardware or licensing fees; and

there would be minimal need to clean or transform the data.

6.2.2 A decentralised data provision approach

An alternative option to a centralised data repository is to require DNSPs to be responsible for processing

requests for consumers’ metering data from authorised third parties on an on-demand basis.

Under this option, retailers could no longer be required to satisfy requests from third parties, thereby

minimising duplication in effort. DNSPs were selected to implement the decentralised approach to minimise

the number of providers with which a third party would need to engage with, if operating across Australia (17,

instead of 30 or more retailers). This approach also ensures that additional costs are not placed on

potentially smaller new entrant retailers, which could create additional barriers to entry and so competition in

the electricity retail market.

Figure 3 provides a diagrammatic representation of the approach to providing data under a proposed

decentralised data provision approach.

Figure 3: Overview of a decentralised data provision approach

Under the decentralised option, third parties would request data from the relevant DNSP. We expect that this

would involve changes to their existing systems so as to accommodate authorised third parties and to meet

the on-demand objective for accessing consumers’ metering data. However, given that distributors already

store consumers’ metering data for other purposes that would not be additional (or perhaps only small)

incremental data storage related costs.


HoustonKemp.com 26

We therefore expect that the incremental costs of a decentralised data distribution approach would be:

project management, consultation with stakeholders, and coordination amongst DNSPs;

the cost of central coordination so that there is consistency in what is implemented by DNSPs; and

each DNSP would need to incur the upfront and ongoing costs of:

> implementing and maintaining a process for data transfer;

> the cost of developing and maintaining a data portal and API; and

> system testing prior to rollout.

In total, we estimate that the decentralised approach option would cost around $3.4 – $12.5 million to

establish, and have an on-going cost of between $0.3 – 2.8 million per year –Table 2. This represents costs

of around $6.8 – $38 million over the next 20 years in present value terms27.

Table 2: Assumed establishment and ongoing cost of a decentralised data provision approach

Establishment costs Ongoing costs per year 28

Project management $117,000-$468,00029 $11,700 – $93,600

Consultation $58,500-$117,000 $5,850 – $23,400

Coordination between DNSPs $117,000-$468,000 $11,700 – $93,600

Verification and data transfer system $78,000 – $312,000 per DNSP30, or $1 – 4

m in total

$7800 – $62,400 per DNSP, or $0.1 – 0.8

$403,600 in total

System changes $26,000 – $104,000 per DNSP, or $0.3 –

1.4 m in total

$2,600 – $20,800 per DNSP, or $33,800 –

$270,400 in total

Data portal $50,000 – $100,000 per DNSP, or $650,000

– $1,300,000 in total

$5,000 – $20,000 per DNSP, or $65,000 –

$260,000 in total

Application programming interface $10,000 – $50,000 per DNSP, or $130,000

– $650,000 in total

$1,000 – $10,000 per DNSP, or $13,000 –

$130,000

Testing $78,000 – $312,000 per DNSP, or $1 – $4

m in total

$7,800 – $62,400 per DNSP, or $0.1 – $0.8

m in total

Total $3.4- 12.5 m $0.3 – $2.8 m

27 This assumes that implementation occurs in 2019 and system ‘go lives’ in 2020. The discount rate used is seven per cent.

28 Ongoing costs assumed 10 to 20 per cent of establishment costs.

29 We have assumed that the cost of project management and consultation is the same as the centralised approach. In addition, we have assumed there is an additional need for coordination, and this would have similar costs to project management.

30 We have assumed that the cost each DNSP incurs to develop the verification and data system, data portal, API and testing is the same as the centralised option. In addition, there would be modification costs associated with upgrading their existing systems. The low case assumes it would take an IT (with a salary $120,000 and associated overhead costs 30 per cent) 2 months to modify DNSPs existing systems. The high case assumes that it would take an external consultant (at twice the cost of an internal staff) 4 months.


HoustonKemp.com 27

The large range in the estimated costs reflect the significant uncertainty regarding what the end

requirements of the system will be, the required capabilities and required changes to existing systems.

Importantly, we have made the following assumptions:

data procedures and definitions would be maintained as part of other processes, such as NEM12 file updates;

the data would be stored using cloud storage, with appropriate data security features;

there is scope to leverage existing systems, and so no need for additional hardware or licensing fees; and

there would be minimal need to clean or transform the data.

6.3 Preliminary recommendation

Our high-level analysis supports a conclusion that a centralised approach to handling data requests and data

delivery would likely be a lower cost option compared to a decentralised approach led by distributors. In

addition, we expect that the centralised approach:

would have lower implementation risks and would be more adaptable over time because of the need to develop and modify only one system rather than multiple systems;

would lower costs to authorised third parties seeking consumers’ data as there is only one system to integrate with; and

could create a national consumer data repository that links account holders with NMIs and metering data, making it easier to access historical data.

That said, a key negative for a centralised approach is the potential lack of innovation in data delivery and

systems that might arise. We expect that the B2B hub would evolve and adapt over time to meet technology

changes and support new and innovative data service delivery requirements of third parties. That said, an a

DNSP led approach could promote further innovations in data delivery and systems as each DNSP attempts

to improve the effectiveness and efficient of their own processes.

A key question for a centralised data repository approach is how best to fund the necessary development

and ongoing maintenance costs. Our preliminary recommendation is for AEMO to be responsible for

administering the system and handling requests, although this could be given to a contracted private

provider over time. AEMO would be well placed to develop such a system given that it already manages the

e-hub and the MSATS systems. Costs would therefore be recovered from DNSPs and retailers through

market fees.


10. Are the estimated costs for development and ongoing maintenance a centralised or decentralised

implementation of the system reasonable?

11. What are reasonable timeframes for implementation under each of the options considered?


HoustonKemp.com 28

A1. Summary of current arrangements for requesting and obtaining

consumers’ metering data

This section sets out the current arrangements of how consumers can obtain their data, either themselves or

via a third party.

Requesting own data Authorised third party

Origin 31

Online form requiring:

Account number

Name

Email

Date of birth

Phone number

Period of time

+ proof of authority

AGL32


Account number

National Meter Identifier (NMI) number

Supply address

Name

Date of birth

Phone number

Email

Period of time

+ Name of authorised representative

+ Name of company

+ Position in company

+ ABN

+ Phone number

+ Signed consent form (emailed)

Energy Australia33


Name

Account number

Postcode

Period of time

Email address

+ Emailed PDF containing:

Name of company

Contact name

ABN

Address of company

Email of company

Phone number of company

Meter number

NMI

Customer’s address

Signed consent form

Energex34


Name

Phone number

Email

NMI

Address

Date from which account was held

Period of time

+ Name of representative

+ Email of representative

+ ABN

+ Name of company

+ Reason for requesting data

+ Signed authority form (scanned)

Essential energy35


Name

Phone number

Email

Address

NMI

+ Email request containing:

Signed letter of authority (preferably on a company letterhead)

31 https://www.originenergy.com.au/meter-data-request.html#/

32 https://refer.agl.com.au/forms/request-metering-data/

33 https://www.energyaustralia.com.au/meterdatausagerequest#/usageRequest

34 https://www.energex.com.au/about-us/contact-us/forms/general-forms/meter-data-request

35 https://www.essentialenergy.com.au/content/meter-data-requests


HoustonKemp.com 29

Period of time

Catcpha

SAPN36

Creation of an online account, the first step of which requires:

Name

Phone number

Email

Recent power bill

+ Emailed PDF requiring:

Company name

ABN

Address

Company phone number

Signed authority form (scanned)

Jemena37

Simple email request, OR

Creation of customer account, the first step of which requires:

Name

Email

Recent power bill

No information provided on website

AusNet Services38

Creation of an online account, the first step of which requires:

Name

Email

Phone number

Pre-registration process via email, then an online form, the first step of which requires representee data

Powershop Uncertain – creation of a customer account to access data No information provided on website

Dodo39

Email request containing:

Account number

Name

Address

Phone number

Email

NMI

Date of birth

Desired method of delivery

+ Representative’s name

+ Representative’s contact number

+ Representative’s email

+ Signed consent form

36 https://www.sapowernetworks.com.au/centric/customers/mymeter.jsp

37 https://electricityoutlook.jemena.com.au/

38 https://meterdata.ausnetservices.com.au/webui/#/

39 https://www.dodo.com/power-gas/quick-links/meter-data-provision-request/

https://electricityoutlook.jemena.com.au/


HoustonKemp.com 30

A2. Summary of workshops held to date

We have held several workshops with stakeholders to date, including:

a whole day workshop with all stakeholders on 15 November 2017 in Melbourne;

and half day workshops in Melbourne with:

> distribution network service providers on 11 December 2017;

> retailers on 11 December 2017; and

> consumer/third party data users on the 12 December 2017

HoustonKemp.com

Sydney

Level 40

161 Castlereagh Street

Sydney NSW 2000

Phone: +61 2 8880 4800

Singapore

8 Marina View

#15-10 Asia Square Tower 1

Singapore 018960

Phone: +65 6817 5010

facilitating access to consumer electricity data · facilitating access to consumer electricity...

Documents