Redundant data collection: more data, faster, and with fewer gaps

To ensure high data quality and reliable collection, Metry supports collection from multiple data sources. Many of our customers choose to add a second data source to their setup. Combining collection from utility provider portals with collection via file transfer is what we call redundant collection – and there are several good reasons to do it.

With an additional data source, you get more data, faster

When Metry collects data from utility providers, we retrieve calculated consumption data. That means we can never get more data than the utility provider has made available. With redundant collection, however, we can reach coverage levels that in some cases exceed what either method delivers on its own.

If you collect data only from utility provider portals or files, you will typically get around 90% of all data when you check mid-month. At the start of the month, coverage will be noticeably lower, depending on the meter type. But if you combine data from both portals and file transfers, you get 98% of all data mid-month and 90% at the start of the month.

In short: more data, and available sooner. But why does this actually happen?

Different collection methods have different strengths

After more than a decade of data collection, we know the capabilities and limitations of each method well. Keeping track of your meters, ensuring regular collection of readings, and handling delayed or updated values are all functions that need to work reliably – and different collection methods handle these in different ways. Here is a closer look at when portal collection is the better choice, when file transfer is, and why combining them produces better results.

Meter inventory

Meters get added and removed. In a large property portfolio, it can be difficult to keep track of all the buildings being acquired and sold, and all the subscriptions being activated and cancelled.

Utility provider portals already contain all the information about which meters you are paying for. With portal-based collection, our automated systems pick up new meters as they appear and flag any meters that appear to have left your organisation. That kind of automatic meter inventory is not possible with file transfers alone. When setting up file-based collection, the utility provider needs to be told which meters to include – and if meters are added or removed, that information has to be updated manually. With portal collection, meter inventory happens automatically.

Regular collection of readings

Changes on utility provider portals can sometimes delay collection – whether because the portal is down for maintenance, or because login credentials have changed and are blocking our automated collection.

Portals can slow collection for other reasons too. For one of our customers, we discovered that the utility provider had decided not to make readings available on their portal until they had manually reviewed all values and approved the invoice. That manual process took considerable time, meaning no values were accessible until it was complete.

File transfers can also stop unexpectedly – for example, when a utility provider updates their systems – and need to be investigated in the same way. That is precisely why combining the two methods matters: if one source goes down, the other provides backup, and you can be confident that your readings will be available in Metry.

Access to hourly data

More and more of our customers add file-based collection specifically to get hourly readings. This is not surprising – more granular data gives a better basis for energy monitoring. Monthly data is sufficient for basic monitoring, but hourly readings open up more targeted ways to understand consumption: identifying and reducing costly power peaks, for example, or working with building automation.

Hourly readings are often technically available from utility provider portals, but experience has shown that file transfer is the better method for this. Collecting hourly readings via portals slows down the overall collection process and increases the risk of delays. Since redundancy is rarely needed at the hourly level, we recommend collecting hourly data via file transfer only – and getting that data faster as a result.

Handling delayed or corrected readings

Readings can unfortunately arrive late or contain errors, and when that happens they need to be updated. With file-based collection, you receive the values that are available at that moment – but delayed values rarely come through automatically. Even when you might expect utility providers to send all new values, we have found that they typically only send the previous day’s or previous month’s readings, not the delayed ones. Delayed values that later become available are picked up directly from the portals by our automated systems.

When it comes to incorrect readings, errors on utility provider portals are uncommon since these are usually connected to the provider’s billing system. But if something does go wrong on a portal, it is considerably simpler to rerun the automated collection to retrieve the corrected values – unlike with file transfers, where you would need to contact the utility provider and request a corrected file.

Make your data collection more secure with redundancy

Whether you want more data faster or more reliable collection overall, redundant collection is a straightforward way to get there.

At Metry, we treat energy data seriously – and for something this important, we think it makes sense to use both belt and braces. Get in touch if you want to know more about our different collection methods.

If you are already a customer, you can see which collection method is active for each meter directly in the portal, and add an additional data source from there.