Why data disappears during imports

The most important thing about an integration is that no data gets lost in the process. In practice, that is rarely how it goes. Missing values are common, and the usual causes – poor implementation and inadequate maintenance – apply to most types of integrations. But when it comes to collecting meter readings, there is a third significant challenge: irregular data flows.

Poor implementation

When an integration is built for the first time, a lot can go wrong – particularly if it is set up by someone who does not have a complete picture of how the two systems work and are expected to interact. Every time new code is written, there is a risk of bugs creeping in. You might get it right on the first attempt, or incorrect data might quietly enter your system in a way that is difficult to detect.

Integrations can either be custom-built by a consultant or an IT manager, or you can use a ready-made, standardised integration. The choice between the two might sound straightforward – who would choose a pair of “standard” trousers over a tailored pair? But that choice is less obvious when it comes to integrations.

Unlike a custom-built integration, a ready-made one is well tested. It has been built by people who know both systems, refined as problems have been discovered, and maintained over time. To continue the tailoring analogy: the ready-made pair has been cut from a well-proven pattern, while the bespoke version has not yet been tested to see whether the fabric holds up or the zip survives a machine wash. With ready-made integrations, you can be confident that the foundational work has been done correctly. And since integrations require ongoing maintenance, it makes sense to take the straightforward problems off the table from the start.

Maintenance is unavoidable

The more integrations you have set up, the more maintenance work you accumulate. Bugs and weaknesses in systems will always need to be addressed – an IT system does not become more secure over time by itself.

Who do you turn to when a bug stops your imports? Or when data goes missing because something has changed in one of the connected systems? You might contact the consultant or IT manager who built the integration – but how are they supposed to know what has changed? Any maintenance or update will cost either staff time or consultancy fees. And there is a real risk of ending up in a situation where responsibility bounces between parties and nobody is willing – or perhaps able – to take ownership of fixing the problem.

Vulnerabilities get discovered, and the underlying systems keep changing. It is a continuous process that makes it impossible to treat any integration as a one-and-done solution.

Irregular data flows make smart integrations harder

Even when implementation has been done correctly and maintenance is in place, values can still go missing during transfer. The reason is the irregular data flows that meter reading collection involves.

Most systems that import meter readings schedule those imports to run once a day. In an ideal world, that would mean all values from the past 24 hours automatically arrive in your chosen system. But meter readings frequently come in late, and if the import logic is built to only pull data from the last 24 hours, any values that arrive outside that window are missed. The same logic creates problems when you want to import historical data.

Delayed, missing, or corrected values also raise the question of how and when estimates and updates should be imported. All of these factors together make the collection and transfer of meter data one of the most technically complex integration challenges there is.

Secure your collection with streamed data and ready-made integrations

To simplify both implementation and maintenance, Metry offers ready-made integrations to the majority of major property management and energy systems, as well as a number of smaller visualisation tools. We take responsibility for maintaining and updating those integrations as needed, and we work closely with our partners to ensure they do the same on their side. When we find a problem, we fix it – and that fix applies to all our customers across all their properties. By reusing integration structures, we can apply the same solution much faster than if each integration had to be handled individually.

For irregular data flows, we have a solution called Metry Streams. With Metry Streams, we push new values as soon as they are registered in Metry. That means all new and updated values reach your energy management system as they come in, regardless of when they arrive or which time period they cover.

An integration is only as good as its data source

At Metry, we work to collect 100% of all data – but there is little point in doing that if values are lost due to poor integrations on the other end. With ready-made integrations, an open API, and a team that is happy to help, connecting Metry to your energy management system is straightforward. Get in touch if you would like some help getting set up.