In an overview of a major Expert Focus Whitepaper for Waterbriefing, Dr Atai Winkler, Principal Consultant at PAM Analytics, dispels some of the misconceptions around analytics, defines and explains some common terms associated with analytics, particularly predictive analytics, and gives examples of how predictive analytics can be used to improve asset management in the water industry.
Atai Winkler: Much has been and is written about how analytics can improve asset management in the water industry. Most of it is high level discussion and describes the outcomes expected of using analytics to improve asset management. Little detail is offered and few if any case studies presented.
Whenever using analytics, it must be remembered that data have their own story to tell and it is up to us to find the meaning of the story - it is not for us to impose our story on the data.
Definitions
Firstly, let’s start with the relative lack of clarity about definitions.
Most if not all articles on the topic use the terms business intelligence, predictive analytics (which includes machine learning), data mining, data science and artificial intelligence interchangeably.
Business intelligence is the simplest form of analytics. Business intelligence systems look backwards to report what has happened but do not provide any learning from this history as to what may happen in the future. They summarise historical data in graphs, dashboards and tables with unusual values or values of interest highlighted.
Predictive analytics is a step-up from business intelligence. It uses historical data to develop models for predicting, simulating, forecasting and optimising what is likely to happen under a range of different scenarios. The models have predictor (explanatory) variables that allow users to gain insight and understanding into the system being modelled, for example asset performance.
The predictor variables in the models must make sense and be able to be explained - models that give good results but do not satisfy these requirements will ultimately prove to be poor models. It is always better to have models that fit the data less well but are understandable as they are good foundations for further model development than models that fit the data better but which cannot be explained.
Before predictive models are developed, exploratory data analysis must be carried out to understand the data fully and prepare them for the modelling. Although these steps take time, their importance cannot be underestimated as they are the basis of successful analytics.
Application to asset management in the water sector
PAM Analytics’ Predictive Asset Management system PAM uses a predictive analytics model (survival analysis) to model, simulate and optimise asset management using work order data, asset data and other data.
An important concept in survival analysis is censored observations (assets in asset management). A censored asset is an asset that did not suffer the event (failure in this case) during the observation period or study. This does not mean that censored assets will never fail, only that they did not fail during the study they will (may) fail at some time after the end of the study but with respect to the study they are regarded as censored assets. If censored assets are omitted from the analysis, the results of any analysis will be biased.
The law of diminishing returns applies when using proactive maintenance to reduce the risk of asset failure because the marginal gain (defined as the increase in survival probability from each additional proactive intervention) decreases as the number of proactive interventions increases. In the limiting case, over maintenance costs money with no gain in asset reliability.
Using pumps as an example, failure data on 1,000 pumps over a ten year period were collected to calculate the pumps’ failure distribution. We want to fit the data to a failure probability distribution to understand how the risk (probability) of pump failure changes as the pumps are used. The figure below shows the failure probability distribution. The hatched area is the probability of surviving at least four years.
Failure Probability Distribution
For a pump to fail at any time after four years, it has to survive for four years - pumps which failed before four years are not available for failure after this period. Therefore, the probability of a pump failing after four years is conditional on it surviving for four years. This conditional probability is called the (instantaneous) hazard rate. It is used to calculate the probability of a pump failing after a specified time, for example four years, from the number of pumps that were in use just before this time.
Cox Proportional Hazards Model
The hazard rate can be used to identify the assets in greatest need of immediate proactive maintenance by using the Cox proportional hazards model to develop an asset survival model. This is a predictive model that uses work order, asset and other data to calculate the cumulative hazard (cumulative risk of failure) of each asset. The cumulative hazard is the sum over time of the instantaneous hazard rates (risks of failure).
The next figure below shows how the asset survival model is used for operational asset management optimisation. The pumps (1,300 wet well submersible pumps) are ranked in descending order of their cumulative hazards.
Cox Proportional Hazards Model for Operational Asset Management Optimisation
The pumps with the highest cumulative hazards and therefore the highest risks of failure are identifiable and it is these pumps that should be considered for immediate proactive maintenance.
Asset Survival Simulation
The asset survival model can be combined with discrete event simulation for strategic asset management optimisation. The optimisation is carried out heuristically by running a large number of simulations to study how a range of factors affect the total asset management cost and then determine the conditions that minimise the total asset management cost subject to operational constraints, for example the attitude to the risk of asset failure and work capacity.
The simulation uses the concept of risk tolerance to help determine when an asset should be replaced. It is the maximum acceptable level of repeated asset failure and is measured on a 5 point scale: 1 = risk averse; 5 = risk tolerant. Risk tolerant asset management policies lead to lower asset maintenance costs than risk averse policies but result in higher asset replacement costs and consequence costs, and so lead to much higher total asset management costs than risk averse polices.
The factors that can be considered in the simulation include the:
- maximum work capacity, i.e. the maximum number of proactive interventions and the maximum number of reactive interventions that can be carried out each month *
- number of consecutive monthly reactive interventions a pump can have for it to be deemed to have failed **
- threshold survival probability for proactive maintenance
- threshold survival probability for reactive maintenance
- cost of proactive maintenance and cost of reactive maintenance by pump class
- cost of replacement pumps
- consequence cost of asset failure
* maintenance capacity constraint
** risk tolerance constraint
For all maintenance capacities, even a small amount of proactive maintenance reduces the total cost, with the extent of the reduction depending on the maintenance capacity and risk tolerance constraints.
At very high risk tolerances and exclusively proactive maintenance, the optimal point has been passed – the assets are over maintained. The optimal asset management policy does not require all the maintenance capacity and so unnecessary costs will be incurred beyond the optimal point.
Condition Monitoring
Different predictive models are required for asset management data and condition monitoring data because the data are recorded at different time levels. Work order data for asset management are recorded at the daily level when work is carried out, and modelled at the weekly or monthly levels whereas condition monitoring data are recorded regularly and at much higher frequencies, for example every 15 minutes.
Condition monitoring data are time series data, i.e. observations at a regular frequency. The parameters must be sensitive to asset condition and leading indicators of asset deterioration. Since condition monitoring is used in near real-time, models for condition monitoring data must be simple and able to be calibrated and updated quickly and easily.
The models should only be used for short-term forecasts because of the high sampling frequency. The forecasts will show if the signal is level or if it is tending towards a limit or if the change in level is permanent indicating possible failure. It is for these reasons that condition monitoring addresses immediate operational issues it does not address tactical and strategic issues.
Data Mining
Data mining is the process of analysing data to find patterns and relationships in the data. It is a ‘walk through the data’ without a particular objective in mind but with an open mind as to the patterns and relationships that may be revealed. This means that data mining is an interactive process not a prescriptive event.
In contrast to predictive analytics, data mining does not require a hypothesis to be specified (in predictive analytics a model is proposed, and then developed and tested). As the name implies, the aim is to find what is in the data before analysing and modelling them in detail.
Common data mining techniques include:
- Anomaly detection: finding unusual data or data errors that require further investigation
- Association rules: identifying events (in time) or actions that tend to occur together
- Clustering: allocating objects to clusters where the objects in each cluster are similar with respect to the data used to define the clusters and the objects in different clusters are different with respect to at least one variable used to define the clusters.
These techniques are only used infrequently in asset management in the water industry.
Data Science
Data science involves a number of skills that although related are usually too many and too broad for one person to have. It does not follow that very good computer programmers or scientists are also very good mathematicians or statisticians, and vice-versa. The skills of each overlap to some extent and are complementary but are not substitutes for one another.
Artificial Intelligence (AI)
The term artificial intelligence was first used in 1955 by John McCarthy, professor of mathematics at Dartmouth College, USA. Since then, many claims and promises about AI have been made. For example, in 1957 the economist Herbert Simon predicted that computers would beat humans at chess within 10 years (it took 40 years) and in 1967 Marvin Minsky, head of the AI laboratory at Massachusetts Institute of Technology (MIT), said that within a generation AI would be ‘created’.
Even though both men were preeminent in their fields, their forecasts were badly wrong. This may explain why many claims of great advances in AI have been and are treated with caution.
AI is very good at predicting outcomes for new data that are similar to the data on which it was trained and which do not contain new information or features. For example, imagine an AI system that translates French documents into English. How would it cope with a new document that contains a German word or sentence? If it had not been trained to recognise German, the translation would be wrong. This example shows why today’s AI is known as narrow or weak AI - it can only perform the particular task it was trained for.
According to Professor Josh Tenenbaum of MIT, data must have structure for AI to extract value from them in the way that the human brain extracts value by understanding structure, and cause and effect.
For example, objects created by humans, for example images and speech, have structure, and so AI can analyse them. AI systems that can reproduce the flexible intelligence of human beings and can solve problems they were not specifically trained for have not yet been developed. The long term aim is to create this type of AI (general or strong AI). Whilst narrow AI can outperform humans at tasks they were trained for, it is expected that general AI would outperform humans at most cognitive tasks.
In October 2019 in an article in Wired Yoshua Bengio, a professor at the University of Montreal, expressed frustration at how companies exaggerate the capabilities of AI. ‘I think it would be a good thing if there's a correction in the business world, because that's where the hype is,’ he wrote. Unfortunately, it appears that many people believe that AI can solve all modelling problems. This capability may be achievable one day but it is not currently available.
The main concern that many people have with AI lies in the middle of the process between the input data and output data its black-box nature, i.e. the models cannot be accessed and therefore understood.
Even if they could be accessed, they can only be understood in very simple cases and so cannot answer fundamental questions that people have when modelling data to help them gain insight and understanding. This is the main problem that many people have with AI and may explain why they find AI difficult to relate to and are reluctant to use.
According to Professor Tommi Jaakkola of MIT (MIT Technology Review, April 2017) it is impossible to understand very large AI systems with thousands of units per layer and maybe hundreds of layers. On the other hand, parametric models can be interpreted and related directly to the system being analysed and modelled.
The black-box approach to modelling was neatly summed up by the Canadian mathematician Alexander Dewdney. He wrote that artificial neural networks have a ‘something-for-nothing quality, one that imparts a peculiar aura of laziness and a distinct lack of curiosity about just how good these computing systems are. No human hand (or mind) intervenes; solutions are found as if by magic; and no one, it seems, has learned anything’.
AI cannot be used to predict failure from work order data for at least two reasons: firstly, it requires very big training datasets far larger than are available; and secondly, it requires perfect data. Neither condition is satisfied with work order data in the water industry. Even if the first condition is satisfied, the data would have to be transformed to new forms that are an accurate representation of reality. However, as described above, AI does not have this functionality.
At a more practical level, it is not good enough for practitioners of asset management to be told that the failure probability of one pump is 0.37 and the failure probability of a similar pump is 0.73 numbers without context are of little value. They would rightly ask ‘why the difference?’. AI cannot answer this key question but predictive analytics can by studying the model to identify reasons for the difference.
AI can be used for forecasting condition monitoring data. The problems associated with using AI to predict failure using work order data do not occur condition monitoring generates a lot of perfect data. Since the input data to the models are previous values of the variable being forecast, the models are relatively simple and can be updated frequently to use the most recent data in near real-time to forecast the next few values. Furthermore, the large number of condition monitoring signals increases the appeal of using AI for this type of data.
Understanding data analytics is key to delivering on increasingly tough performance targets
To sum up, the use of data analytics for asset management in the water industry has massive potential to deliver the intelligence the water companies are looking for in order to optimise their operational processes and manage their risks. However, at the moment the lack of clarity around the meanings of a number of terms used, sometimes rather loosely, in analytics is potentially obscuring, rather than highlighting, the considerable benefits it has to offer.
While business intelligence is already used extensively because it presents summary historical information on asset performance, lack of understanding of predictive analytics means the sector as a whole is still failing to maximise the benefits it could achieve by using predictive analytics to optimise asset management by minimising the risk of asset failure and gaining insight and understanding into asset performance.
For companies that really get to grips with how predictive analytics and AI can be used to significantly improve asset management, the insights and understanding they gain will undoubtedly help them to meet the increasingly tough targets in service and performance they are now expected to deliver on an ongoing basis.
Click here to download Exploring the potential of analytics for asset management in the water industry in full
Contact :Dr Atai Winkler
Principal Consultant
PAM Analytics
This e-mail address is being protected from spambots. You need JavaScript enabled to view it
07817 263016