The promise of big data is at last starting to emerge for gas and electricity customers, but sadly only with regulatory intervention.

Several years ago, when big data started to proliferate it was welcomed as a great leveller.  It would help businesses understand how to provide for different segments of their markets.  In turn that would ensure consumers got the best deals.

New insights to understand mass behaviours would help consumers.  Big data would give businesses and service providers the insights to tailor their offers to people’s individual needs.

But arguably for gas and electricity customers the opposite has been true.

For sure, savvy and promiscuous customers have been able to surf between the bargains that energy providers have offered.

Meanwhile, big data helped identify the two segments of customer whose behaviour has not been price sensitive.  Those on pre-pay arrangements for whom switching is very difficult.  The others who may know they could do better elsewhere but stick with expensive standard rates anyway.

In effect, big data identified the two groups of people that have had to subsidise the lower-cost deals that the price surfers have enjoyed.

Regulatory pressure to nudge two thirds of people off the standard rates will present new challenges for energy suppliers.  Margins will come under more pressure at the same time as they need to communicate to their customers why they should remain loyal.

That will present a new challenge for big data too, to extract the real customer insights needed to win them over.