Predicting Output Variables


The Accuracy and Business Value of Machine Learning to Assess Water Main Infrastructure

By Doug Hatler

Municipal and investor-owned utilities often rely on subjective criteria to determine which water mains in their system need replacing. To add to the inaccuracy, the decisions are often made when opportunity strikes, like where street paving will happen in the near future. This process leads to inefficient spending of limited resource dollars, leaving utilities extremely vulnerable to financial and structural risk.

This is unwanted pressure for utilities, as they’re facing increased state and federal regulatory pressures to create efficient, comprehensive asset management plans.

Asset management practices combined with Artificial Intelligence, specifically Machine Learning, provide a new method for assessing the condition of buried water mains. Specifically, AI and Machine Learning allow utilities to properly align maintenance, rehabilitation, and replacement strategies to better allocate limited resources.

Digital Condition Assessment Using AI and Machine Learning
Machine Learning-based condition assessment tools are relatively new, but are now commercially available. Machine Learning, a category of AI, provides computers the ability to learn without being programmed. It uses automated and iterative models to learn about patterns in big data, detecting anomalies and identifying a structure that may be new and previously unknown. Through this capability, Machine Learning supports a new way of aligning maintenance and, in turn, asset management planning by creating more accurate analysis despite using less data.

Fracta offers a fast, accurate and affordable digital condition assessment solution to predict the Likelihood of Failure (LOF) of water distribution mains.

Fracta is also fully integrated with Esri’s market-leading ArcGIS software. The integrated Fracta and Esri platforms provide an architectural framework to readily integrate with other important software applications used by water utilities such as Enterprise Asset Management (EAM), Computerized Maintenance Management Systems (CMMS), and Hydraulic Modeling.

Many utilities, public utility commissions, and consulting engineers still view the Fracta Machine Learning and GIS approach as a “black box,” as happens with any new technology, with the primary concern being accuracy.

Balanced Accuracy for LOF Predictions
The data that comes out of a Machine Learning model is only as accurate as the data that goes into the model (i.e., “garbage in, garbage out”). Fracta uses a Supervised Machine Learning model with input variables (x) and an output variable (Y). The model uses an algorithm to learn the mapping function from the input to the output, Y = f(x). The goal is to approximate the mapping function so well that for new input data (x) the algorithm can predict the output variables (Y) for that data.

It is supervised learning because the process of algorithm learning from the training dataset is similar to a teacher supervising the learning process. By using a training data set with the correct answers known, the algorithm iteratively makes predictions on the training data and is corrected by the teacher. Learning stops when the algorithm achieves an acceptable level of performance. Generally, 80% of the historical data are used to train, and 20% are used to validate the Machine Learning model.

Assessing a water distribution system with a Machine Learning model requires an understanding of both failures and non-failures. True Positive Rate (TPR) measures the proportion of correctly identified real positives. In the Fracta solution, that means correctly identifying the high probability failures. True Negative Rate (TNR) measures the proportion of correctly identified actual negatives. This methodology focuses on correctly identifying the segments that have low LOF. The accuracy is a balance between high LOF and low LOF results. Because the training and validation of the model are based on 80% of the data, the maximum Balanced Accuracy that it can achieve is 80%.

The Next Generation of Condition Assessments:  Fast, Accurate and Cost Effective
Traditionally, condition assessments of buried water mains typically fall into two categories: indirect and direct. An indirect desktop study method should always occur first. A direct or physical inspection and condition assessment are accurate for the pipe tested, but it tends to be slow, very expensive, and labor intensive. Multiple physical measurements are required for correlation and confirmation. The results are difficult to extrapolate to system-wide recommendations, which could be based on arbitrary assumptions and weights (i.e., older pipes are more in need of replacement than newer pipes).

A performance-based buried infrastructure asset management approach involves a detailed inventory by pipeline segment and monitoring how well individual pipelines are meeting the level of service that is required of them. Since buried infrastructure is not readily accessible, performance-based management of these buried assets has historically not been performed in the water industry.

A more robust approach would be a large-scale comparison of various factors to generate a more refined and accurate prediction-based assessment on the disparate interactions between component variables. Machine Learning has emerged as a technology to make a significant impact in buried water infrastructure asset management. Machine Learning consumes large, complex data sets containing more variables than what humans can process with current tools. This objective, data-driven method overcomes human limitations with their inherent subjectivity and biases and provides more accurate results that help utilities make better replacement decisions.

Due to the large amount of historical and geospatial data needed to run Machine Learning algorithms, water main condition assessments contain all the necessary components of an ideal application for water utilities. Pipe data and the surrounding environmental data covering installation year, pipe material, break history, pressure class, geographical location, elevation, pipe diameter, proximity to other infrastructure systems, and soil composition can all be taken into consideration while also assessing hundreds of other variables unique to a specific utility and pipe location. Consistently analyzing this data can uncover trends, gain insight on pipeline health, and offer data-driven assessments.

Doug Hatler

New pipe data strengthens the predictive power of a Machine Learning algorithm. Machine Learning can also benefit utilities with a limited asset or breakage data by “filling in the gaps.” Machine Learning can utilize many streams of data to perform certain predictions and begins to learn patterns that can inform situations where many of the common data points may not be available creating a new digital revolution in advanced asset management practices.  The more data a model contains, the more robust the model. As utilities are constantly collecting data such as new breaks and installed pipes, that data can continually be fed into a Machine Learning model.

In February 2019, Fracta launched its next wave of capabilities. They couple its fast, accurate, and affordable LOF predictions with Consequence of Failure (COF) to calculate a monetized Business Risk Exposure (BRE) and an estimated replacement cost for every buried water main in a distribution system. Fracta COF determines the consequences, or severity, of the failure.

Utilities can calculate the BRE in terms of risk ranking and direct and indirect costs. This approach gives an objective assessment and translates the results into financial terms that water engineers, planners, and finance professionals can use to make fast, accurate and capital-efficient risk mitigation decisions about buried water main infrastructure.

Incorporating a Machine Learning condition assessment like Fracta into a proper infrastructure and asset management program will enable utilities to meet the Modified Approach under GASB 34 for reporting the value of buried water mains.  This will contribute to a more accurate accounting of the value of the assets.  It also contributes to the reduction of economic impacts incurred from water main breaks and more efficient allocation of funding by water utilities. Use of best practices and a more accurate, objective tool will align maintenance and capital repair and replacement strategies to more efficiently leverage scarce financial and human resources.  They also inject financial integrity and accountability to the planning process and refine the investment strategy so a utility will be in a better position to defend planning efforts and justify pipe replacement projects.

Doug Hatler is Chief Revenue Officer at Fracta.