Using Big Data to empower energy consumers

Sophie Adams

Sophie Adams

Publication date

Industry sector

Supply and Retail Market

Data. It’s the buzzword powering the energy revolution - from the smart home and connected new products, services and business models broadening consumer choices, to the network innovations transforming industry infrastructure.

But how are we using it as a regulator to deliver for consumers?

At Ofgem, the insight afforded by Big Data is key to our work in building a smarter, fairer and more flexible energy system.

Only through advances in data can we proactively develop policies or principles that will harness the benefits and potential savings for consumers from transformations in the sector.

Only through data can we identify and put in place protections for those who may struggle to engage with the ‘smart’, more digitalised future, so they don’t get left behind – particularly people in more vulnerable circumstances.

And only through data can we intervene speedily to stop potentially harmful behaviours.

But it’s not just about how we use data ourselves. We have an enabling role too. Through our Data Services unit, we’ve been taking important steps.

A cross-sector approach

We face a system with more renewable energy, more community based generation, more storage, more electric vehicles. New consumer propositions, like bundled services, may cross sector boundaries which means consumers, industry and regulators themselves will need to collaborate in new ways.

To enable this, data needs to be accessible and digestible so that consumers are empowered to make the best choice for them.

Consumers also need to be considered more holistically in terms of how they engage with utilities in their everyday lives, not just as separate energy, water or telecoms customers.

To keep pace, we are looking at jointly evolving data services across regulated industries. It will mean we need data principles, to define how industry should treat regulatory data, and encourage accessibility and sharing of information while keeping data secure at all times. We need to enable where possible and protect where necessary, across all regulatory functions.

In practice, we’re already leading and learning from pioneering joint Big Data initiatives. Our ‘midata in energy’ project is enabling consumers to share their data with trusted third-party firms to find an energy solution that suits them best. While our switching trial programme, the ‘energy customer database’, seeks to highlight those disadvantaged by a lack of market engagement and make it easier for them to shift onto better solutions that will benefit them. We’ll be sharing results of the database trial later this year.

Meanwhile, from this we’re building a strong platform for constructive cross-sector engagement on how we can reduce the burden of exchanging data with regulatory bodies in general, and gaining a strong understanding of the data pressure points industry face on a daily basis. One common issue is that industry often submits duplicate data to multiple government and consumer bodies, while at the same time we struggle with old systems to manipulate the data.

Challenging ourselves

While we want industry to sign up to better ways of working with data, we’re also seeking to lead by example.

We’ve been reviewing our data needs and information requests, and building a more streamlined, centralised internal data service aligned to the consumer outcomes we aim to deliver.

We’ve also developed an internal virtual data hub and exchange interface, and are trialling visualisation tools to build insights from multiple data sets at the touch of a button. This will not only benefit the work of our internal data users, but will mean we can be more agile in responding to market changes to the benefit of consumers.

Next steps

Big Data cannot be developed in isolation, and we are far from the end of our journey. We will be engaging more closely than ever with stakeholders and other regulators, and want to be as open as possible in sharing our learnings and progress. If you want to hear more about our approach, please get in touch at