A semi-automated approach to validation and error diagnostics of water network data

ABSTRACT

We propose a method for quality assurance of raw data from water distribution networks in near real-time. Well-known and novel data analysis methods, including a timestamp drift test, are combined to produce a malfunction indicator database for diagnosing anomalies within data acquisition practices. The method was applied to 112 flow and 111 pressure data sets, covering on average 32 months, located throughout the distribution networks of three Danish utilities. Around 10% of measurements in the utilities’ meter data sets were absent and 3–35% were categorized as dubious or erroneous. The most common types of anomalies for flow and pressure data were flatline and time stamp inconsistencies. Time drifts were identified in all three utilities and a similarity analysis revealed a simultaneous occurrence of many anomalies. These high rates could have been avoided if the proposed method had been implemented to automatically highlight meter errors and system-wide problems in data collection.

KEYWORDS:

• Data validation
• error diagnostics
• water supply

Acknowledgements

We thank all participating partners of the LEAKman project, Halsnæs Forsyning and Martin Brandt-Ewon from Nordvand A/S for providing data and answering questions related to data management practices.

Disclosure statement

No potential conflict of interest was reported by the authors.

Supplemental Material

Supplemental data for this article can be accessed here.

Additional information

Funding

This work was supported by the LEAKman project and partners under the Danish Eco-Innovation Program (MST-141-01277/NST-404-00378).