Infrastructure maintenance and replacement optimization under multiple uncertainties and managerial flexibility

Abstract

Infrastructure maintenance and replacement decisions are subject to uncertainties such as regular asset degradation, structural failure, and price uncertainty. In the engineering domain, Markov Decision Processes (MDPs) typically focus on uncertainties regarding asset degradation and structural failure. While the literature in the engineering domain stresses the importance of addressing price uncertainties, it does not substantiate the observations of such uncertainties through optimization modeling. By contrast, real option analyses (ROAs) that originate from the financial domain address price uncertainties but generally disregard asset degradation and structural failure. Accordingly, this piece of current research brings both domains closer together and proposes an optimization approach that incorporates the flexibility to choose between multiple successive intervention strategies, regular asset degradation, structural failure and multiple price uncertainties. A practical result of the current research is a realistic approach to optimization modeling in which state space reduction is achieved by combining prices into portfolios. The current research obtains transition probabilities from existing price data. This approach is demonstrated using a case study of a water authority in the Netherlands and confirms the premise that price fluctuations may influence short-term maintenance and replacement decisions.

Keywords:

• Maintenance
• replacement
• optimization
• Markov Decision Process
• real options analysis

Introduction

Many infrastructure assets are ageing and reach the end of their useful life. Infrastructure asset owners are confronted with large scale replacements in the coming decennia (Park et al. 2012, Hall et al. 2016). Huge capital expenditures are involved and the planning and financing of these investments is an issue of great concern (Haffner and Gennady 2011, Power et al. 2016). The American Water Works Association states in a press release: “Renewal and replacement of infrastructure and financing for capital improvements top the list of water industry concerns for the 4th year running” (AWWA 2019). The US Federal Highway Agency warns for unprecedented challenges: “Ageing roads and bridges that carry greater traffic volumes and heavier loads than ever need extensive rehabilitation” (FHWA 2019). Similar concerns are shared among infrastructure asset owners worldwide (Stewart 2001, Klatter et al. 2009, Orcesi 2016).

Ageing infrastructure imposes the need for maintenance and replacement optimization (Frangopol 2011, Elcheikh and Michael 2017). Such optimization modeling is challenging for several reasons. First, acknowledging that uncertainties exist, the identification of the main uncertainty drivers is difficult given that such drivers can be related to the infrastructure integrity, the environment surrounding the infrastructure, and the costs associated with preserving the functions of the infrastructure (Ilg et al. 2017, Sinha et al. 2017, Lange 2018).

A second challenge is the identification of alternatives. Costs, uncertainty, changing societal demands and long technical life cycles of infrastructure assets favor lifetime extension by maintenance without compromising safety (Lange 2018). Generally, several activities with different cost and risk profiles are available to meet the required performance demands. Activities include, for example, regular maintenance, overhauls, major overhauls, and renewals as well as the numerous sequential combinations of these activities.

A third challenge is that uncertainty drives the need for managerial flexibility. A decision-maker will monitor uncertainty drivers and base future decision on the development of those drivers. This managerial flexibility has value that should be incorporated into replacement optimization (Lander and Pinches 1998, De Neufville and Scholtes 2011, Cardin et al. 2015).

The literature, which is reviewed in the following section, offers maintenance and replacement optimization approaches but none of them covers the aforementioned challenges in an integrative manner while in practice these challenges are often found in combination. More specifically, optimization approaches found in the literature have difficulties with quantifying uncertainty which is reflected in the substantiation of underlying transition probabilities and, price uncertainty is generally omitted.

Therefore, the aim of the current research is the development of an integrative approach towards infrastructure replacement optimization, which includes regular asset degradation, structural integrity, price uncertainty, multiple sequential options, and managerial flexibility. Such a model supports decision-making in practice and is interesting from a scientific perspective as it demonstrates the importance of addressing price uncertainty.

The outline of this article is as follows. The following section presents a literature review on the inclusion of uncertainty regarding asset integrity and prices in maintenance and replacement optimization modeling, which identifies the research gap. Hereafter the structure of the model is presented, including the motivation of the states, actions, transition probabilities, and rewards for the current research objective. The subsequent section presents in-depth motivation on the approach to modeling price uncertainty in relation to asset deterioration and structural failure. Hereafter the model is demonstrated in a case study, which is followed by a discussion and conclusions.

Literature review

The following literature review is structured along the lines of two key observations relevant for an integrative approach to maintenance and replacement optimization under uncertainty. The first observation is that uncertainty regarding asset integrity is often modeled with transition probabilities that are difficult to substantiate in practice when condition data are unavailable. A second observation is that price uncertainty is rarely addressed in maintenance and replacement optimization despite its importance as stressed by researchers.

Uncertainty regarding asset integrity

Optimizing multiple sequential intervention strategies under uncertainty leads to probabilistic approaches to maintenance and replacement optimization. A mainstream approach for optimizing sequential decision-making is the Markov Decision Process (MDP) (Frangopol et al. 2004, Baik et al. 2006, Kolobov and Kolobov 2012, Lin et al. 2019). An MDP is a stochastic serial decision model that incorporates uncertainty and the managerial flexibility to optimize future decisions or actions (Puterman 1994, Frangopol et al. 2004, Kolobov and Kolobov 2012). MDPs are widely used in the literature to address asset integrity uncertainties in bridge and pavement systems with respect to the maintenance and replacement optimization. For example, MDPs for optimizing bridge deck and road maintenance are presented by Costello et al. (2005), Robelin and Madanat (2007), Faddoul et al. (2011), and Oliveira et al. (2017). Similar, MDP approaches have been developed for wastewater systems (Wirahadikusumah et al. 1999, Baik et al. 2006, Lin et al. 2019).

In these MDP approaches, transition probability matrices define the probabilities for transferring from one condition state to other condition states. Moreover, the impact of maintenance activities on condition improvement is estimated.

Standardized visual inspections for bridge and pavement systems and CCTV assessments for sewer pipes provide such condition data, generally on a 1 to 5 scale. Transition probabilities are derived from this data. As an example, a dedicated solution for wastewater systems is proposed by Baik et al. (2006), who propose an ordered probit model which estimates the transition probabilities as increments in condition based on a discretization of a continuous deterioration function. Condition data are required to validate such an approach.

Another approach to obtain transition probabilities from inspection data is proposed by Elcheikh and Michael (2017) who suggest a PERT-distribution and Monte Carlo simulation to estimate the likelihood that an asset is in one of the condition states. Although the authors have a different modeling objective their approach for estimating the likelihood of an asset being in a certain condition state can be used to establish transition probabilities.

Nevertheless, when data are unavailable or unsuitable, the difficulties associated with estimating transition probabilities and the impacts of maintenance actions on the condition, limit the application of these MDP approaches. Lethanh et al. (2017) identify several difficulties in data-driven approaches to find such probabilities. Challenges include the possible lack of data, inconsistencies with the data, and possible biases in the data. In response, solutions to deal with inadequate data are proposed such as mechanistic-empirical approaches (Lethanh et al. 2017), genetic algorithms (Almeida et al. 2015, Compare et al. 2015) or specific adaptation of the modeling to available data (Adey et al. 2018). Fuzzy sets theory, a rule-based expert knowledge system, is also known to be supportive to incorporate uncertainty (Elcheikh et al. 2013, Masteri et al. 2018). Mohanta et al. (2005), for example, propose a fuzzy logic approach to establish transition probabilities in a Markov model for maintenance scheduling. Here, the knowledge of experts is used to define membership functions to address uncertainty.

Not all infrastructures have visible and measurable condition states like gravity sewers and pavements. For example, pump systems, the machinery of movable bridges or locks, swing bridges in wastewater treatment plants and aeration facilities fail occasionally, are repaired and put into service again. In these situations, deterioration is generally expressed as a rate of occurrence of failures (Ascher 2007).

Price uncertainty

Another observation is that most probabilistic MDP models for maintenance and replacement optimization do not address price uncertainty. Faghih Sayed Amir and Kashani (2018) and Swei et al. (2017) observe that although price fluctuations have an impact on future maintenance, rehabilitation and construction, such knowledge is generally ignored by researchers in the engineering domain. Similar conclusions are drawn by Mirzadeh et al. (2014) and Yu and Ive (2011), who emphasize the importance of proper assessments of price developments with respect to the Swedish road infrastructure and the UK construction industry, respectively. Rehan et al. (2016) and Younis et al. (2016) equally emphasize the importance of a proper assessment of price indices to forecast capital expenditures. These authors stress that unit prices and producer price indices are specific for sectors and geographical locations.

Additionally, Ilbeigi et al. (2017) observe that even previous research devoted to forecasting future prices and cost indices does not quantify price uncertainty.

Despite the observation that price uncertainty is generally ignored in MDP models for maintenance and replacement optimization, price uncertainty is addressed in the domain of real options analysis (ROA). ROA stems from financial options. An option gives a holder the right but not the obligation to exercise a financial transaction at a future date. This right has value, the so-called option value (Black and Scholes 1973, Merton 1973, Cox et al. 1979, Brealey et al. 2017).

In a ROA, price uncertainty is often modeled as a Geometric Brownian Motion (GBM) (Mun 2006, Guthrie 2009, Younis et al. 2016, Farida Agustini et al. 2018). A GBM is a stochastic simulation process in which a price follows a random walk. Such random walk can be simulated with a Monte Carlo simulation or represented as a binomial lattice (Cox et al. 1979).

ROA applications dedicated to the maintenance and replacement optimization of ageing infrastructure are scarce but good ROA applications are found in related fields. Several authors have developed ROA applications for optimizing investment decisions in hydropower and flood defense while addressing price uncertainty influenced by climate change scenarios (Woodward et al. 2014, Kim et al. 2017). Another ROA application focused on investment decisions is presented by Martani et al. (2018) who optimize urban real estate investments under uncertain future rental scenarios. Several ROA applications address risk allocation and regulation in public–private partnerships. In this field, Liu et al. (2017) use ROA to establish a price mechanism for termination prices of public–private partnership projects. Other ROA models focus on an optimal (win-win) risk allocation of revenues and price cap regulation in public–private partnerships (Pellegrino et al. 2011, Carbonara and Pellegrino 2018). Power et al. (2016) consider buyout, revenue sharing and minimum revenue guarantee as options in transportation public-private partnerships. ROA is also known to be a good instrument for valuing concessionaires in road toll projects (Ford et al. 2002, Feng et al. 2015, Buyukyoran and Gundes 2018).

An ROA application for the valuation of operation and maintenance contracts for gravity sewer systems is presented by Park et al. (2012). Interesting about this research for the current objective is that an asset deterioration function is coupled with maintenance costs. A dedicated infrastructure replacement optimization ROA, given one variable for price uncertainty, is presented by Van den Boomen et al. (2018).

Similar to MDP, the application of ROA has its challenges and its critiques. In contrast to MDPs, ROAs often do not address asset deterioration, nor multiple price uncertainties. A discrete approach to ROA will become complex when multiple uncertainties are involved. Like MDP, ROA quickly suffers from state explosion. For this reason, ROA applications found in the literature dealing with multiple uncertainties generally use a Monte Carlo simulation to combine several uncertainties into one and assume constant volatility. However, such approach provides limited insight in how distinct uncertainties contribute to the result.

Summarizing the literature, MDPs are emphasized in the engineering domain when considering asset integrity uncertainty, whereas price uncertainty is generally ignored. By contrast, ROAs emphasize price uncertainty, while generally ignore asset integrity. Moreover, both approaches quickly fall prey to state explosion. The current research aims to develop an integrative approach that incorporates a sequence of intervention strategies, multiple price uncertainties, structural integrity, and regular asset degradation.

Model formulation

The current research uses an MDP to model the optimization problem because an MDP optimizes sequential decisions under uncertainty (Puterman 1994, Frangopol et al. 2004, Hillier and Lieberman 2010). The current approach is a finite MDP but truncates the time-variant final states with a discounted value representing all expected future life cycle costs belonging to the final decision. Value iteration is used to find the optimal sequence of managerial decisions in each state. Groundwork explaining MDPs and algorithms to find optimal policies are provided by Puterman (1994) and Kolobov and Kolobov (2012). The value iteration algorithm is fed with transition probability matrices which contain the transition probabilities for transferring from each state to another for each action.

The following subsections first describe the modeling approach and motivate the choices made for the states, actions, transition probabilities and rewards. This approach is visualized in Figure 1. The subsequent section motivates the in-depth approach to the modeling of price uncertainty, ageing and structural integrity. This modeling is the input for establishing transition probabilities and rewards.

Figure 1. MDP approach for the current model.

States

The state space is defined by five discrete and finite variables, namely, asset type, time, age, the price level of the operational expenditures (OPEX) and price level of the capital expenditures (CAPEX). The state-space contains the Cartesian product of these variables.

The asset types are an existing asset, an overhauled asset, a major overhauled asset, and a renewed asset. Time is a second variable, and the current modeling is nonstationary. Each asset has a certain age at a certain time and is bounded by a maximum life. The fourth and fifth variables are the levels for OPEX and CAPEX. These are discrete variables derived from forecasts based on analyses of long-term historic price indices as will be demonstrated in the case study section.

As an example, a state-designated as $[\text{overhauled asset},\mathrm{ }t,\mathrm{ }n,\mathrm{ }{d}_{\mathrm{opex}},\mathrm{ }{d}_{\mathrm{capex}}]$ depicts an overhauled asset, $n$ years after the overhaul ($n$ = relative age) in the $t\mathrm{th}$ year from now ($t$ = absolute time). The overhauled asset consequently has been maintained for $n$ years. The last two variables define the price levels in the $t\mathrm{th}\mathrm{ }$year which will be explained in the section on price uncertainty modeling. The OPEX and CAPEX price levels follow a binominal lattice and their positions are given by respectively $d_{\mathrm{opex}}$ and $d_{\text{capex }}$ number of down moves in year $t.$

Actions

The four allowable actions depending on the state are to maintain, overhaul, a major overhaul or renew the infrastructure, as depicted in Figures 1 and 2.

To maintain denotes providing regular maintenance and minimal repair. Such a strategy is often referred to as “minimal repair” or “as good as old” (Rigdon and Basu 2000, Ascher 2007).

To overhaul refers to substantially larger maintenance efforts such as revisions, rehabilitations, or partial replacements. An overhaul prolongs the remaining service life and may positively impact the failure rate. This strategy is also referred to as “imperfect repair” as it brings the asset to a better condition but not “as good as new” (Rigdon and Basu 2000). A major overhaul refers to a complete renovation, refurbishment, or reinforcement of the infrastructure. All these actions are substantial and will significantly prolong the remaining service life of the infrastructure and lower the failure rate (Pascual et al. 2006, Ascher 2007, Shafiee et al. 2011). Renewal designates a full replacement. Such a strategy is denoted as “perfect repair” or “as good as new” (Rigdon and Basu 2000, Ascher 2007).

Regular maintenance is valid for an existing asset, an overhauled asset, and a major overhauled asset. However, inherent in this action is the probability for structural failure, that is exceedance of a limit state. In such an event, the consequence is a penalized renewal because an unplanned renewal is generally more expensive than a planned renewal.

There is a sequence in possible intervention strategies. An existing asset can be maintained, overhauled, major overhauled and renewed, whatever is optimal under conditions like age and price levels for OPEX and CAPEX. An overhauled asset can be maintained, major overhauled, or renewed. A major overhauled asset can be maintained or renewed. Finally, a renewal brings the asset to a truncation state that estimates all time-variant expected future life cycle costs (infinite) after the action is taken.

A simplified representation of the MDP model is provided in Figure 2, whereby the four states that represent the asset type (the grey circles), contain the states for time, age, OPEX, and CAPEX price levels.

Figure 2. Simplified representation of the actions and states of the MDP model.

Transition probabilities

Transition probabilities designate the probability of transferring from one state to another when a certain action is taken. The transition probabilities involved are the probabilities for arrival at a certain OPEX and CAPEX price level after an action is taken and the probability of a structural failure or reaching a limit state.

In contrast to MDP applications found in the literature such as Wirahadikusumah et al. (1999), Faddoul et al. (2011), and Oliveira et al. (2017), who use condition data to estimate transition probabilities to move from one condition state to another, the current study proposes to express condition degradation in maintenance repair costs substantiated with a failure rate. The current study is focused on assets which occasionally fail and are allowed to fail, and for which condition data are scarce or unavailable. The subsequent uncertainty quantification of maintenance costs can be derived from existing price data.

Price uncertainty is modeled with underlying discrete binominal lattices which represent all possible price paths as explained in detail in the following section. Illustrative visualization of the action maintains with probabilities for one time step is presented in Figure 3. A maintenance action results in the following probable outcomes. One decision epoch further ensures that the asset will be one-time epoch older if no structural failure occurs ($1-P_{fs}$). In such a case, the aged asset will be subject to four probable price level states, specifically, the price level of the operational expenditures will go up (${\eta }_{\mathrm{OPEX},\mathrm{up}}$) or down (${\eta }_{\mathrm{OPEX},\mathrm{down}}$) and the price levels of the capital expenditures will go up (${\eta }_{\mathrm{CAPEX},\mathrm{up}}$) or down (${\eta }_{\mathrm{CAPEX},\mathrm{down}}$). The fifth probable outcome is that the asset will reach its limit state (threshold for a structural failure with probability $P_{\mathrm{sf}}$). In that case the model truncates with a time-variant corrective renewal including all estimated future life cycle costs.

Figure 3. Schematized visualization of the action maintains and probabilities for transferring to successive states.

In each state, the appropriate OPEX prices are traced, based on the decision to maintain, and assessed against the possible CAPEX prices of the alternative CAPEX decisions, including all future life cycle costs. The actions, overhaul, major overhaul and renewal (CAPEX actions), will result in a transfer to another asset type with relative age 0 at the moment the action is taken. Hereafter maintenance starts again, for the other asset type.

The current model assumes all CAPEX prices follow the same price path directions. For example, when prices for overhauls are high at a certain time, the expectation is that renewal prices are proportionally high as both investment actions are related to the development of market construction prices.

In the current modeling, this assumption does not hold for operational expenditures when transferring from one asset type to another as a consequence of a CAPEX decision because a new asset type can have a totally different OPEX cost profile from the previous one (demonstrated in the case study section). Accordingly, the OPEX lattices of each asset type are not expected to be connected only by up or down moves. Rather, all possible nodes one decision epoch further are considered when transferring from one OPEX lattice to another.

Rewards

The immediate rewards of each action transferring from one state to another directly follow from the underlying OPEX or CAPEX cash flow lattices.

However, special attention is required regarding the rewards when transferring to the truncation state. This is a probable outcome of the maintenance action in the case of a structural failure or a direct outcome of the renewal action. The truncation value is a time-variant estimate of perpetuity of future life cycle costs of renewal as proposed by Guthrie (2009) and Van den Boomen et al. (2018). In the case of a structural failure, this truncation value is additionally penalized with a factor 1.5 for the first investment because an unplanned renewal is more expensive than a planned renewal.

Modeling price uncertainty, ageing and structural failure

The previous section formulates the overall MDP approach to optimize a sequence of maintaining, overhauling, and replacing decisions while taking multiple price uncertainties, ageing, and structural failure into account. The current section motivates the approach to assess the price uncertainties, ageing, and structural failure.

Price uncertainty

A common convention is to model price uncertainty using geometric Brownian motion (GBM) (Mun 2006, Guthrie 2009, Younis et al. 2016, Farida Agustini et al. 2018). A GBM is a continuous stochastic simulation process where prices follow a lognormal distribution with drift μ and a normally distributed shock ∊ with a volatility σ. The constants drift $\mu$ and volatility $\sigma \mathrm{ }$are estimated based on historic prices by using the mean and standard deviation of the log-returns of the prices (Mun 2006, Guthrie 2009, Francis and Kim 2013).

A GBM can be expressed as a binomial lattice as in Figure 4, which is the discrete form of the continuous stochastic simulation process. The magnitude of its up move $U$ and down move $D$ are defined by volatility $\sigma$ and conform to Equations (1)(1) $$U=\hspace{0.17em}\text{exp}\hspace{0.17em}(\sigma )$$(1) and (2)(2) $$D=1/U=\hspace{0.17em}\text{exp}\hspace{0.17em}(-\sigma )$$(2) (Cox et al. 1979) as following: (1) $$U=\hspace{0.17em}\text{exp}\hspace{0.17em}(\sigma )$$(1) (2) $$D=1/U=\hspace{0.17em}\text{exp}\hspace{0.17em}(-\sigma )$$(2)

Figure 4. Cash flow development in a linear binomial lattice.

In Figure 4, $X(d,t)$ represents the cash flow at decision epoch $t,$ having experienced $d$ number of down moves.

Cox et al. (1979) propose to use drift $\mu$ and volatility $\sigma$ to derive the actual probability ${\theta }_u$ for an up move and ${\theta }_d$ for a down move, respectively, according to Equations (3)(3) $${\theta }_u=\frac{1}{2}+\frac{1}{2}\frac{\mu }{\sigma }$$(3) and (4)(4) $${\theta }_d=1-{\theta }_u$$(4) as following: (3) $${\theta }_u=\frac{1}{2}+\frac{1}{2}\frac{\mu }{\sigma }$$(3) (4) $${\theta }_d=1-{\theta }_u$$(4)

As the current research will introduce asymmetric option payoffs, estimating the actual probabilities, as in Equations (3)(3) $${\theta }_u=\frac{1}{2}+\frac{1}{2}\frac{\mu }{\sigma }$$(3) and (4)(4) $${\theta }_d=1-{\theta }_u$$(4) , is not sufficient (Copeland and Antikarov 2003, Guthrie 2009). Rather, the interest lies in the certainty equivalent probabilities, designated as risk-neutral probabilities for up and down moves, that is, ${\eta }_u$ and ${\eta }_d,$ respectively. To obtain the risk neutral probabilities from the market data, the certainty equivalent of the capital asset pricing model is followed in Equations (5)(5) $${\eta }_u=\frac{K-D}{U-D},$$(5) and (6)(6) $${\eta }_d=1-{\eta }_u,$$(6) (Guthrie 2009) as follows: (5) $${\eta }_u=\frac{K-D}{U-D},$$(5) (6) $${\eta }_d=1-{\eta }_u,$$(6) where ${\eta }_u$ is the risk-neutral probability of an up move; ${\eta }_d$ is the risk-neutral probability of a down move; and $K$ is the risk-adjusted growth factor of the expected cash flows.

The actual probabilities ${\theta }_u$ and ${\theta }_d$ are correlated to the risk-neutral probabilities ${\eta }_u$ and ${\eta }_d,$ respectively, by the risk-adjusted growth factor $K,$ which, for one decision epoch to the next, is defined as (Guthrie 2009) following: (7) $$K=\frac{{\theta }_u\times X_u+{\theta }_d\times X_d}{X_0}-\text{market}\text{risk}\text{premium}\times {\beta }_{\text{CAPM}}$$(7)

The first term on the right side of Equation (7)(7) $$K=\frac{{\theta }_u\times X_u+{\theta }_d\times X_d}{X_0}-\text{market}\text{risk}\text{premium}\times {\beta }_{\text{CAPM}}$$(7) represents the expected growth factor of an underlying market price variable where $X_0$ is the price of this price variable at the beginning of the time step, $X_u=\mathrm{ }{X}_{0}U$ and $X_d=\mathrm{ }{X}_{0}D.$ The second term represents the well-known market risk premium and the ${\beta }_{\mathrm{CAPM}},$ and subtracts a risk premium from the expected growth of the underlying market price variable.

Anticipating the modeling of the current research, two direct equations are required to arrive at the predicted cash flows and their probabilities for each node $(d,t),$ where $d$ is the number of down moves and $t$ is the time. The cash flow $X(d,t)$ follows from: (8) $$X(d,t)=X(0,0)\hspace{0.17em}\text{EXP}\hspace{0.17em}\left((t-2d)\sigma \right),$$(8) and the overall probability for arriving at any node $(d,t)$ from node $(0,0)$ is as following: (9) $$P(d,t)=\left(\begin{array}{c}t\\ d\end{array}\right)\times {\eta }_u^{t-d}\times {\eta }_d^d,$$(9) where $\left(\genfrac{}{}{0pt}{}{t}{d}\right)=\frac{t!}{d!\left(t-d\right)!}$ represents the standard binomial coefficient. Figure 5 graphically represents the relation between the stepwise risk-neutral probabilities and the overall probabilities for arriving a node $(d,t)$ for two-time steps.

Figure 5. Risk neutral probabilities and overall probabilities for arriving at node (d,t) in a stationary binomial lattice.

Modeling price uncertainty for each price in a ROA separately with its own binominal lattice, as depicted in Figures 4 and 5, will quickly lead to state explosion as $k$ prices result in ${(t+1)}^k$ future possibilities for price levels. This illustrates the major challenge for modeling price uncertainty in a ROA and leads the current research to another domain in finance, namely, the portfolio theory. The portfolio theory was introduced by Markowitz (1952) and targets at diversification of risk by combining financial securities into a single portfolio. The portfolio theory of Markowitz (1952) allows for combining these financial securities in a single portfolio with a new portfolio drift ${\mu }_p$ and portfolio volatility ${\sigma }_p.$ The portfolio mean ${\mu }_p\mathrm{ }$is defined by Equation (10)(10) $${\mu }_p=\sum _{i=1}^k{w}_i\times {\mu }_i,$$(10) as follows: (10) $${\mu }_p=\sum _{i=1}^k{w}_i\times {\mu }_i,$$(10) where $k\mathrm{ }$is the number of constitutions in portfolio $p;$ $w_i$ is the proportional weight of constitution $i$ in the portfolio; and ${\mu }_i$ is the mean of constitution $i.$ The portfolio volatility ${\sigma }_p\mathrm{ }$is defined by Equation (11)(11) $${\sigma }_p=\sqrt{\sum _{i=1}^k\sum _{j=1}^k{w}_i\times w_j\times {\sigma }_{ij}},$$(11) as follows: (11) $${\sigma }_p=\sqrt{\sum _{i=1}^k\sum _{j=1}^k{w}_i\times w_j\times {\sigma }_{ij}},$$(11) where ${\sigma }_{ij}=\mathrm{COV}\left(ij\right);$ ${\sigma }_{ii}=\mathrm{VAR}\left(i\right)={\sigma }_i^2;$ and ${\sigma }_{jj}=\mathrm{VAR}\left(j\right)={\sigma }_j^2.$ Equation (11)(11) $${\sigma }_p=\sqrt{\sum _{i=1}^k\sum _{j=1}^k{w}_i\times w_j\times {\sigma }_{ij}},$$(11) expresses portfolio volatility as a function of the proportion of constitutions in a portfolio, the separate volatilities of the constitutions, and the correlations of the constitutions.

The GBM and binominal lattice are founded on log-returns of prices and arithmetic mathematical operations over time, while the portfolio theory builds on compounded simple returns (the non-log version of returns), mathematical geometric operations over time and arithmetic operations over assets (Francis and Kim 2013). For a strict interpretation, a proper mathematical conversion from log return to simple return values is required when combining both theories. These mathematical transformations follow the relationship: $\text{log return value}=\ln (1+\mathrm{simple}\text{return value}).$

The portfolio theory allows for merging different market prices and their underlying binominal lattices in a single portfolio binominal lattice. This significantly reduces the required state space in the current ROA modeling.

Asset degradation

A challenge in the current modeling is the ageing of repairable infrastructure. In the absence of sufficient condition and failure data, the current research proposes an approach which builds on failure rates and maintenance repair costs. A similar line of reasoning is proposed by Park et al. (2012) for a situation where condition data are limited.

The power law process is a popular model applied in practice for ageing repairable assets (Ascher and Feingold 1984, Rigdon and Basu 2000, Ascher 2007). As such, it provides a relationship for the rate of occurrence of failures, here designated as $\lambda \left(n\right)$ in Equation (12)(12) $$\lambda (n)=\frac{\beta }{\theta }{\left(\frac{n}{\theta }\right)}^{\beta -1},$$(12) as follows: (12) $$\lambda (n)=\frac{\beta }{\theta }{\left(\frac{n}{\theta }\right)}^{\beta -1},$$(12) where $n$ is the asset age and $\beta ,\theta$ are shape and scale parameters, respectively, that are derived from failure data or expert judgment.

While ageing leads to more repairs and affects operational expenditures, it does not affect the development of market prices of the constitutions defined by ${\mu }_i\mathrm{ }$and ${\sigma }_i.$ It will, however, affect how many units of a certain price constitution are used. Therefore, ageing directly affects the weights $w_i$ in a portfolio. This imposes a difficulty as it introduces time variance into the portfolio binominal lattice. The time-variant portfolio lattice characteristics as defined by Equations (1)(1) $$U=\hspace{0.17em}\text{exp}\hspace{0.17em}(\sigma )$$(1) through (7)(7) $$K=\frac{{\theta }_u\times X_u+{\theta }_d\times X_d}{X_0}-\text{market}\text{risk}\text{premium}\times {\beta }_{\text{CAPM}}$$(7) are derived from ${\mu }_p(t)$ and ${\sigma }_p(t)$ for each time step, but they cannot be used to describe the move from one decision epoch to another in the combined portfolio binominal lattice. This is because these characteristics describe the next move under the assumption that each characteristic remains unchanged, which is not the case. Rather, the expected price at a successive decision epoch depends on the constitutions’ price developments in the market and the proportion in which they are used, not on the current levels and proportions of operation and maintenance expenditures.

For this reason, the approach followed herein uses the estimated future market prices (Equation (8)(8) $$X(d,t)=X(0,0)\hspace{0.17em}\text{EXP}\hspace{0.17em}\left((t-2d)\sigma \right),$$(8) ) and their overall arrival probabilities (Equation (9)(9) $$P(d,t)=\left(\begin{array}{c}t\\ d\end{array}\right)\times {\eta }_u^{t-d}\times {\eta }_d^d,$$(9) ) as constraints and derives the corresponding combined portfolio risk-neutral probabilities ${\eta }_{\mathrm{cp},u}\left(d,t\right)$ and ${\eta }_{\mathrm{cp},d}\left(d,t\right)=1-\mathrm{ }{\eta }_{\mathrm{cp},u}\left(d,t\right)$ as detailed in Equation (13)(13) $${\eta }_{cp,u}(d,t) = \{\begin{array}{l}\frac{P_p(0,t+1)}{P_p(0, t)}\hspace{1em}\mathrm{i}\mathrm{f} d=0\\ \\ \frac{P_p(0, t+1)+P_p(1,t+1)+\dots +P_p(d,t+1)-(P_p(0,t)+P_p(1,t)+\dots +P_p(d-1,t))}{P_p(d,t)},\\ \text{otherwise}\end{array}$$(13) . Although this approach has similarities with the implied binomial tree approach as introduced by Rubinstein (1994), it differs in that Rubinstein (1994) only had the future exercise prices as constraints, while the current research has all future prices as constraints. The application and validity of Equation (13)(13) $${\eta }_{cp,u}(d,t) = \{\begin{array}{l}\frac{P_p(0,t+1)}{P_p(0, t)}\hspace{1em}\mathrm{i}\mathrm{f} d=0\\ \\ \frac{P_p(0, t+1)+P_p(1,t+1)+\dots +P_p(d,t+1)-(P_p(0,t)+P_p(1,t)+\dots +P_p(d-1,t))}{P_p(d,t)},\\ \text{otherwise}\end{array}$$(13) are demonstrated in Supplemental material. (13) $${\eta }_{cp,u}(d,t) = \{\begin{array}{l}\frac{P_p(0,t+1)}{P_p(0, t)}\hspace{1em}\mathrm{i}\mathrm{f} d=0\\ \\ \frac{P_p(0, t+1)+P_p(1,t+1)+\dots +P_p(d,t+1)-(P_p(0,t)+P_p(1,t)+\dots +P_p(d-1,t))}{P_p(d,t)},\\ \text{otherwise}\end{array}$$(13)

Structural integrity

Structural integrity denotes the probability of a total collapse or end of life failure. The European reference design code for new structures (NEN-EN 1990) prescribes a reliability index ${\beta }_R$ as a design parameter for structures, which is also referred to as limit state. The reliability index is derived from load-resistance interference (Leira 2013, Sánchez-Silva and Georgia-Ann 2016). The probability of structural failure is defined by $p_{\mathrm{sf}}=\Phi \left(-{\beta }_R\right),$ where $\Phi$ denotes a cumulative normal standard distribution.

In the Netherlands, mandatory renewed structural safety assessments are required as soon as infrastructures reach their design life or in the case of reconstructions (NEN 8700 2011). Reliability indices for disapproval levels range from 1.8 to 3.3 corresponding with failure probabilities of 0.036 to 4.83E-04, respectively.

Case study

This case study builds on a case study presented in Van den Boomen et al. (2019). The current modeling acknowledges multiple price uncertainties, asset degradation following a power law process and structural failure following interference theory. The inclusion of multiple uncertainties requires a different and more complex model setup and follows the approach described in the previous sections.

The case study is an old pumping station with diesel engines owned by a water authority. There are still a significant number of such diesel pumping stations in the Netherlands. These diesel engines are replaced by electrical engines upon a major overhaul. Although diesel is a relatively cheap source of energy, diesel engines run on fossil energy, are labour-intensive in their maintenance and require specialized maintenance engineers. The question of interest is to optimize this replacement from a cost perspective to support a wider managerial decision framework. The approach followed is applicable to similar case studies that deal with multiple successive options, several price uncertainties, asset degradation modeled with a failure rate and a probability for structural failure.

Cost data

The cost data obtained in Table 1 belong to a pumping station with a capacity of 4200 L/min and a head of 2 m. The cost data are based on registered historic costs and future estimates by maintenance engineers.

Table 1. Cost data in Euros.

	CAPEX	OPEX
Asset type	Construction	Labour	Gasoline	Electricity
Existing asset		75,000	125,000
Overhauled asset	350,000	75,000	125,000
Major overhauled asset	4,000,000	50,000		240,000
Renewed asset	15,000,000	10,000		225,000

Financial market parameters

The risk-free interest rate, market risk premium and ${\beta }_{\mathrm{CAPM}}$ are obtained from a study of the Dutch Authority for Consumer and Market (ACM 2017) and provided in Table 2.

Table 2. Financial market parameters.

Risk-free interest rate	0.83% per year
Market risk premium	4.98% per year
CAPM $\beta$	0.71

Price uncertainty

The drift and volatility of the prices are obtained from historic prices between 1996 and 2018 and presented in Table 3. A clear explanation of how to obtain drifts and volatilities from market prices assuming a GBM is provided by Francis and Kim (2013), Mun (2006), and Guthrie (2009). The Dutch Central Bureau for Statistics (CBS) provides aggregated historic price indices for civil engineering works among which is included construction for water projects (CBS Stateline 2018). The historic price indices for maintenance (labour), gasoline and electricity are obtained from CROW (2018). Conversion from log return values to simple return values is required for the following application of the portfolio theory (Francis and Kim 2013). Table 3 illustrates that conversion is only significant for higher drifts and volatilities (approximately >10%).

Table 3. Annualized drift and volatility derived from historical price indices 1996–2018.

		CAPEX	OPEX			Reference
Parameters		Construction	Labour	Gasoline	Electricity
Drift^a	μ	0.03	0.03	0.06	0.01	Data
Volatility^a	$\sigma$	0.03	0.02	0.27	0.13	Data
Drift^b	μ	0.03	0.03	0.06	0.01	Conversion
Volatility^b	$\sigma$	0.03	0.02	0.32	0.14	Conversion

^a Based on log return values.

^b Based on simple return values.

Figure 6 displays the historic price developments of the cost components showing energy costs to be more volatile than construction costs and labour.

Figure 6. Historic development of price indices for construction, labour, gasoline and electricity.

Anticipating the combining of maintenance costs in two portfolios, the covariance of labour and gasoline and the covariance of labour and electricity is derived from the log-returns of the prices and presented in Table 4. A positive covariance indicates that the log-returns of the prices move in the same direction, whereas a negative covariance indicates an opposite movement. Again, for precision, a transformation to simple return-based values is performed.

Table 4. Covariances of operational prices.

Covariance	Log return	Simple return
COV (labour, gasoline)	8.33E − 05	8.33E − 05
COV (labour, electricity)	−3.88E − 05	−3.98E − 05

Failure rate and structural failure

In the absence of sufficient long-term failure data, the current research, in addition to trend analysis, used a pragmatic expert judgment approach for estimating the scale and shape parameters for β and θ of the power-law process as depicted in Equations (14)(14) $$\beta =\frac{\hspace{0.17em}\text{ln}\hspace{0.17em}(k_j/k_i)}{\hspace{0.17em}\text{ln}\hspace{0.17em}(t_j/t_i)}+1$$(14) and (15)(15) $$\theta ={\left(\frac{\beta }{k_j}\right)}^{\frac{1}{\beta }}{\left(t_j\right)}^{\frac{\beta -1}{\beta }}$$(15) in the following text: (14) $$\beta =\frac{\hspace{0.17em}\text{ln}\hspace{0.17em}(k_j/k_i)}{\hspace{0.17em}\text{ln}\hspace{0.17em}(t_j/t_i)}+1$$(14) (15) $$\theta ={\left(\frac{\beta }{k_j}\right)}^{\frac{1}{\beta }}{\left(t_j\right)}^{\frac{\beta -1}{\beta }}$$(15) where $k_j$ is the expected number of failures in year $j;$ $k_i$ is the expected number of failures in year $i;$ $t_j$ is year $j;$ and $t_i$ is year $i.$

Expert judgment from the water authority was also used to estimate the maximum remaining service lives ($N$) of different asset types. Such estimates are based on a maximum allowable number of failures per year with respect to the impact on society. For example, a pumping station that fails too often may result in negative publicity and reputation damage for a water authority. All data are presented in Table 5 and Figure 7 graphically depicts the estimated failure rates if the asset types are retained for their maximum remaining service lives.

Figure 7. Estimates of priority 1 failure rates for the aged assets when successively maintained for their maximum remaining service lives.

Table 5. Parameter estimation for β and θ based on expected future failures per year.

	Max $N$ (years)	$k_0$(#)	$k_{\left(\frac{2}{3}\right)N}$(#)	$k_N$(#)	$\beta$(−)	$\theta$(−)	$p_{sf}$(−)
Existing asset	5	3	4	5	2.7	3.1	0.005
Overhauled asset	10	2	3	5	3.7	5.7	0.005
Major overh. asset	15	1	3	5	2.7	4.8	0.0005
Renewed asset	80	Truncation with discounted perpetuity of life cycle costs

The last column in Table 5 provides the rounded estimated probabilities for a structural failure (non-repairable) based on the legally demanded reliability indices for existing structures in the Netherlands.

Combined portfolio lattices

The data in the previous sections are used to establish the combined OPEX and CAPEX binominal lattices for the price developments of each asset type. The CAPAX lattices are straightforward as they are built on aggregated construction price indices. The parameters that describe the CAPEX binominal lattices are presented in Table 6.

Table 6. Parameters describing the binominal lattice of construction prices.

Parameters	Symbol	Construction	Reference
Drift (log return)	$\mu$	0.03	Data
Volatility (log return)	$\sigma$	0.03	Data
Up move	$U$	1.03	Equation (1)(1) $$U=\hspace{0.17em}\text{exp}\hspace{0.17em}(\sigma )$$(1)
Down move	$D$	0.97	Equation (2)(2) $$D=1/U=\hspace{0.17em}\text{exp}\hspace{0.17em}(-\sigma )$$(2)
Actual probability up	${\phi }_u$	0.88	Equation (3)(3) $${\theta }_u=\frac{1}{2}+\frac{1}{2}\frac{\mu }{\sigma }$$(3)
Risk-adjusted growth factor	K	0.99	Equation (7)(7) $$K=\frac{{\theta }_u\times X_u+{\theta }_d\times X_d}{X_0}-\text{market}\text{risk}\text{premium}\times {\beta }_{\text{CAPM}}$$(7)
Risk neutral probability up	${\eta }_u$	0.35	Equation (5)(5) $${\eta }_u=\frac{K-D}{U-D},$$(5)

Future CAPEX prices follow Equation (8)(8) $$X(d,t)=X(0,0)\hspace{0.17em}\text{EXP}\hspace{0.17em}\left((t-2d)\sigma \right),$$(8) with the volatility for construction prices as depicted in Table 6. An illustration in Table 7 depicts the first 5 years of the probable development of the future prices of an overhaul.

Table 7. Price development of an overhaul with characteristics from Table 6 in Euros (reference Equation (8)(8) $$X(d,t)=X(0,0)\hspace{0.17em}\text{EXP}\hspace{0.17em}\left((t-2d)\sigma \right),$$(8) ).

d|t	0	1	2	3	4	5
0	350,000	361,913	374,232	386,970	400,142	413,762
1		338,479	350,000	361,913	374,232	386,970
2			327,337	338,479	350,000	361,913
3				316,562	327,337	338,479
4					306,141	316,562
5						296,064

The OPEX lattices are not straightforward because distinct operational prices are combined in a single binominal lattice and because ageing (the failure rate) causes the proportions of price constitutions to change with time. As explained in the previous sections, we propose an implied binomial tree approach to construct a time-variant combined binominal lattice. While the difficulty shifts to the construction of combined binominal lattices, hereafter it allows for using the convenient characteristic of recombining branches in the MDP model, which significantly reduces state space.

Two forces drive the development of maintenance costs, namely, ageing, which is age-dependent, and market price development, which includes inflationary forces and is time-dependent.

First, ageing is considered. For ageing, price increases are related to the failure rate, which is a function of the age of the asset after an action is taken. This age is designated as $n$ with a maximum remaining life of $N.$ Price increases caused by ageing are modeled to conform to the power-law process: (16) $$X(n)=X_0+C_f\left(\frac{\beta }{\theta }\right){\left(\frac{n}{\theta }\right)}^{\beta -1}$$(16) where $X(n)$ are yearly maintenance costs at age $n$ at the current price level, $X_0$ are the maintenance costs just after the instalment of an asset at the current price level, $C_f$ are the costs of a priority 1 failure, $\beta$ and $\theta$ are the shape and scale parameters describing the power law process and $n\mathrm{ }$is the age.

The second operational cost component is energy, specifically, diesel for the existing and overhauled asset and electricity for the major overhauled asset. As no immediate changes are foreseen in energy consumption for each asset type, the expected price of diesel at the current price level remains constant. The impact of price increases and uncertainty caused by inflationary market forces are accounted for when establishing the binominal lattices.

As an example, the proportion of maintenance expenditures and energy expenditures at the current price level are calculated for the existing asset and combined to forecast future prices (Tables 8 and 9). A detailed calculation example for the values depicted in Tables 8 and 9 is provided as Supplemental material.

Table 8. Parameters describing the distinct binominal lattices that combine operational prices for labour and gasoline at current price levels.

Parameters	Symbol							Reference
Age	$n$	0	1	2	3	4	5
Cost labour	$C_1(n)$	75,000	76,277	79,176	83,352	88,657	95,000	Equation (16)(16) $$X(n)=X_0+C_f\left(\frac{\beta }{\theta }\right){\left(\frac{n}{\theta }\right)}^{\beta -1}$$(16)
Cost gasoline	$C_2(n)$	125,000	125,000	125,000	125,000	125,000	125,000	Table 1
Cost total	$C_T(n)$	200,000	201,277	204,176	208,352	213,657	220,000
Weight labour	$w_1$	0.38	0.38	0.39	0.40	0.41	0.43
Weight gasoline	$w_2$	0.63	0.62	0.61	0.60	0.59	0.57
Drift portfolio^a	${\mu }_p$	0.05	0.05	0.05	0.05	0.05	0.05	Equation (10)(10) $${\mu }_p=\sum _{i=1}^k{w}_i\times {\mu }_i,$$(10)
Vol. portfolio^a	${\sigma }_p$	0.20	0.20	0.19	0.19	0.19	0.18	Equation (11)(11) $${\sigma }_p=\sqrt{\sum _{i=1}^k\sum _{j=1}^k{w}_i\times w_j\times {\sigma }_{ij}},$$(11)
Drift portfolio^b	${\mu }_p$	0.05	0.05	0.05	0.05	0.05	0.04	Conv.
Vol. portfolio^b	${\sigma }_p$	0.18	0.18	0.18	0.17	0.17	0.17	Conv.
Up move	$U_p$	1.20	1.20	1.19	1.19	1.19	1.18	Equation (1)(1) $$U=\hspace{0.17em}\text{exp}\hspace{0.17em}(\sigma )$$(1)
Down move	$D_p$	0.83	0.84	0.84	0.84	0.84	0.85	Equation (2)(2) $$D=1/U=\hspace{0.17em}\text{exp}\hspace{0.17em}(-\sigma )$$(2)
Actual prob. up	${\phi }_{u,p}$	0.63	0.63	0.63	0.63	0.63	0.63	Equation (3)(3) $${\theta }_u=\frac{1}{2}+\frac{1}{2}\frac{\mu }{\sigma }$$(3)
Risk growth fac.	$K_p$	1.03	1.03	1.03	1.03	1.02	1.02	Equation (7)(7) $$K=\frac{{\theta }_u\times X_u+{\theta }_d\times X_d}{X_0}-\text{market}\text{risk}\text{premium}\times {\beta }_{\text{CAPM}}$$(7)
Risk neutral prob. up	${\eta }_{u,p}$	0.53	0.53	0.53	0.53	0.53	0.53	Equation (5)(5) $${\eta }_u=\frac{K-D}{U-D},$$(5)

^a Based on simple return values.

^b Based on log return values.

The bottom part of Table 8 provides the characteristics of six separate binominal lattices that are now returned to one binominal lattice using the implied binominal lattice approach explained in the modeling section and shown in Table 9. Equation (8)(8) $$X(d,t)=X(0,0)\hspace{0.17em}\text{EXP}\hspace{0.17em}\left((t-2d)\sigma \right),$$(8) provides the expected prices, Equation (9)(9) $$P(d,t)=\left(\begin{array}{c}t\\ d\end{array}\right)\times {\eta }_u^{t-d}\times {\eta }_d^d,$$(9) provides the expected overall probabilities for arrival at each node, and Equation (13)(13) $${\eta }_{cp,u}(d,t) = \{\begin{array}{l}\frac{P_p(0,t+1)}{P_p(0, t)}\hspace{1em}\mathrm{i}\mathrm{f} d=0\\ \\ \frac{P_p(0, t+1)+P_p(1,t+1)+\dots +P_p(d,t+1)-(P_p(0,t)+P_p(1,t)+\dots +P_p(d-1,t))}{P_p(d,t)},\\ \text{otherwise}\end{array}$$(13) is used to derive the non-stationary risk-neutral probabilities ${\eta }_{u,\mathrm{cp}}\left(d,t\right)$ of the combined lattice as a function of time and down moves.

Table 9. Lattice characteristics for combined labour and gasoline prices (combined portfolio).

d|t	0	1	2	3	4	5
Price development with characteristics from Table 8 (reference Equation (8)(8) $$X(d,t)=X(0,0)\hspace{0.17em}\text{EXP}\hspace{0.17em}\left((t-2d)\sigma \right),$$(8) ).
0	200,000	240,862	291,028	351,128	421,769	503,340
1		168,197	204,176	247,944	300,190	361,481
2			143,243	175,082	213,657	259,603
3				123,632	152,068	186,438
4					108,233	133,894
5						96,158
Overall probabilities for arriving at the cash flows (reference Equation (9)(9) $$P(d,t)=\left(\begin{array}{c}t\\ d\end{array}\right)\times {\eta }_u^{t-d}\times {\eta }_d^d,$$(9) ).
0	1.00	0.53	0.28	0.15	0.08	0.04
1		0.47	0.50	0.40	0.28	0.18
2			0.22	0.35	0.37	0.33
3				0.10	0.22	0.29
4					0.05	0.13
5						0.02
Risk neutral probabilities for an up move ${\eta }_{u,cp}(d,t)$ (reference Equation (13)(13) $${\eta }_{cp,u}(d,t) = \{\begin{array}{l}\frac{P_p(0,t+1)}{P_p(0, t)}\hspace{1em}\mathrm{i}\mathrm{f} d=0\\ \\ \frac{P_p(0, t+1)+P_p(1,t+1)+\dots +P_p(d,t+1)-(P_p(0,t)+P_p(1,t)+\dots +P_p(d-1,t))}{P_p(d,t)},\\ \text{otherwise}\end{array}$$(13) )
0	0.5312	0.5304	0.5290	0.5271	0.5249	n.a
1		0.5304	0.5290	0.5271	0.5249	n.a
2			0.5290	0.5271	0.5249	n.a
3				0.5271	0.5249	n.a
4					0.5249	n.a
5						n.a

Comparing the recombined risk-neutral probabilities in Table 9 with the nonrecombined risk-neutral probabilities in Table 8 reveals no significant differences for the current case study in the first 5 years. However, over time or with higher volatilities and increased failure rates, these differences become more explicit.

Transition probabilities and rewards

The nonstationary OPEX transition probabilities are obtained from the lattices built in the first step of the modeling and are dependent on the state, which is a function of asset type, time, age, down move position for OPEX prices and down move position for CAPEX prices (i.e. Table 9).

The CAPEX transition probabilities up and down are straight forward because these lattices are stationary (i.e. Table 6). Rewards at position $(d,t)$ are similarly obtained from the preconstructed cash flow lattices (OPEX, i.e. Table 9) or directly calculated from CAPEX using Equation (8)(8) $$X(d,t)=X(0,0)\hspace{0.17em}\text{EXP}\hspace{0.17em}\left((t-2d)\sigma \right),$$(8) .

Optimization

The model is programmed in Matlab, and the optimal policies are found by using value iteration (Puterman 1994). The value iteration algorithm uses the transition probability matrices of the four actions (maintain, overhaul, major overhaul, and renewal), their corresponding rewards and the risk-free discount rate.

The maximum service lives of the existing, overhauled, major overhaul and renewed assets are resp. 5, 10, 15, and 80 years for the case study as depicted in Table 5.

The value iteration algorithm calculates the optimal decision in each state. The case study has approximately 80,500 states resulting from the combinations of the state variables: asset type, time, age, price level OPEX, and price level CAPEX. One hundred iterations are performed with an error rate of 4.4E-7 and a calculation time of fewer than 10 s.

Results

The result of the calculations is a table that depicts the best decision for each state along with the discounted costs of all optimal future actions from that state forward. This includes the calculation of optimal service lives corresponding with future best decisions under future circumstances for all price levels, failure rates, and probabilities for structural failures.

Table 10 presents the states of the first 5 years of the current asset, thus providing management information for the immediate decision. The optimal decision for the existing asset is found in the last column of Table 10. The best current decision is to maintain the existing asset until at least year 4. In year 4, the best decision depends on the development of prices. When the operational expenditures of the existing asset are in lattice position $\left(d,t\right)$ = (0,4) while the capital expenditures are in lattice position (2,4), (3,4), or (4,4), the best decision is to overhaul the existing asset. Although prices cannot be compared individually (as all optimal future decisions are incorporated as well), Tables 7 and 9 provide an intuitive indication that an overhaul would be optimal in this situation. Table 9 reveals that the operational expenditures have risen to €421,769 at $(d,t)$ = (0,4), whereas the costs of a major overhaul (Table 7) at nodes $(d,t)$ = (2,4), (3,4) and (4,4) have decreased to €350,000, €327,337 and €306,141.

Table 10. Results for the existing asset.

Existing asset	Time	Age	Down move OPEX	Down move CAPEX	Optimal PV of life cycle cost	Optimal decision
States	0	0	0	0	−26,935,771	Maintain
	1	1	(0, 1)	(0, 1)		Maintain
	2	2	(0, 1, 2)	(0, 1, 2)		Maintain
	3	3	(0, 1, 2, 3)	(0, 1, 2, 3)		Maintain
	4	4	0	0	−30,059,775	Maintain
	4	4	0	1	−28,781,539	Maintain
	4	4	0	2	−27,570,686	Overhaul
	4	4	0	3	−26,433,680	Overhaul
	4	4	0	4	−25,368,884	Overhaul
	4	4	(1, 2, 3, 4)	(0, 1, 2, 3, 4)		Maintain
	5	5	(0, …, 5)	(0, …, 5)		Overhaul

In year 5, at the maximum remaining life of the existing asset, all states indicate an overhaul to be the optimal decision.

If a decision-maker wants to look further, a state can be traced into the future. For example, state $[\text{exisiting asset},\mathrm{ }t=4,\mathrm{ }n=4,\mathrm{ }{d}_{\mathrm{OPEX}}=0,d_{\mathrm{CAPEX}}=2]$ indicates an overhaul. This would transfer the existing asset one decision epoch further ($t\mathrm{ }=\mathrm{ }5$) to a 1-year-old ($n=1$) overhauled asset. The possible down moves for the new OPEX lattice position are 0,1, 2, 3, 4, or 5. The possible down moves for the new CAPEX lattice are up (2 + 0) or down (2 + 1). This results in 12 probable subsequent states, which are depicted in Table 11. In each of these states, the optimal decision would be to maintain the overhauled asset for another year. Similarly, all states of interest can be traced.

Table 11. Results of a 1-year-old overhauled asset in year 5.

Overhauled asset	Time	Age	Down move OPEX	Down move CAPEX	Optimal PV of life cycle cost	Optimal decision
States	5	1	0	2	−30,022,802	Maintain
	5	1	1	2	−29,169,492	Maintain
	5	1	2	2	−28,339,758	Maintain
	5	1	3	2	−27,693,206	Maintain
	5	1	4	2	−27,233,080	Maintain
	5	1	5	2	−26,915,606	Maintain
	5	1	0	3	−28,789,422	Maintain
	5	1	1	3	−27,981,822	Maintain
	5	1	2	3	−27,172,566	Maintain
	5	1	3	3	−26,531,112	Maintain
	5	1	4	3	−26,071,809	Maintain
	5	1	5	3	−25,754,408	Maintain

Navigating through the table of results (i.e. Table 10), based on realistic data, demonstrates that optimal decisions are dependent on price developments.

Discussion

The inclusion of uncertainty in maintenance and replacement optimization is an area under investigation. Frangopol (2011), Ilg et al. (2017), and Scope et al. (2016) classify core challenges as identifying the right sources of uncertainty and selecting appropriate methods to incorporate uncertainties. These authors also conclude that uncertainty modeling remains case-specific depending on type of data and quality of data.

A source of uncertainty which is still overlooked in maintenance and replacement optimization is price uncertainty (Park et al. 2012, Rehan et al. 2016, Younis et al. 2016, Ilbeigi et al. 2017). An important direction for further research is price forecasting, quantifying price uncertainty and the integration of these in maintenance and replacement optimization modeling.

Adding price uncertainty quickly complicates existing MDP maintenance and replacement optimization models because such models fall prey to state explosion. As a solution, Monte Carlo simulation could be applied to merge multiple uncertainties into one aggregated uncertainty (Pellegrino et al. 2011). However, the current research opted for an analytical solution which allows for tracing distinct price uncertainties in separate portfolios.

The current research adds to the body of knowledge on several points. First, it includes multiple price uncertainties in an MDP approach. State explosion is prevented by merging several market prices into portfolios for operational and capital expenditures by using portfolio theory (Markowitz 1952, Francis and Kim 2013).

Second, price uncertainty is applied to actions like regular maintenance, overhaul, major overhaul and renewal whereas the action regular maintenance incorporates repairs and a probability for reaching a limit state resulting in a subsequent unplanned renewal. Condition improvement is achieved after an overhaul, major overhaul or renewal.

The proposed approach follows a line of reasoning presented by Park et al. (2012) who integrate a condition deterioration rate in a ROA application to value operation and maintenance expenditures for sewer systems. Hereafter, these authors apply a sensitivity analysis to account for uncertainty in the condition deterioration rate. Instead of condition rate the current study uses failure rate. Both studies deal with a situation were limited data on asset deterioration are available. Further, the current study optimizes a chain of actions.

Although the current approach includes multiple probabilistic price uncertainties, it does not incorporate probabilistic uncertainty on condition deterioration. When sufficient condition data are available, such uncertainty is often modeled with transition probability matrices, which express the probability of transferring from one condition state to another (Wirahadikusumah et al. 1999, Baik et al. 2006, Lethanh et al. 2017). Depending on the availability of data and type of data, other approaches to uncertainty modeling of condition deterioration exist such as Monte Carlo simulation (Elcheikh and Michael 2017, Lin et al. 2019), fuzzy logic techniques (Masteri et al. 2018) and sensitivity analysis (Park et al. 2012, Scope et al. 2016).

The current study deals with infrastructure for which condition data are not available and failure data are scarce. Therefore, the current research proposes a deterministic approach to address asset deterioration related uncertainty (Scope et al. 2016, Ilg et al. 2017) combined with the probabilistic approach to model price uncertainties. An interesting direction for future research is the inclusion of a probabilistic uncertainty modeling of the failure rate. Such direction could follow a Non-Homogeneous Poisson distribution (Rigdon and Basu 2000). The consequence, however, is the addition of states to the already impressive state space. In such case, it would be interesting to investigate the applicability of more advanced algorithms to find optimal policies in large MDP models (Puterman 1994, Kolobov and Kolobov 2012).

Alternatively, partial observable MDP (POMDP) is a direction to proceed when limited data on condition states or failure rates are available. In a POMDP a decision-maker cannot fully observe the states nor the impact of decisions. A decision-maker will use his beliefs and update his believes when information becomes available (Faddoul et al. 2011, Walraven and Spaan 2018).

Another important direction for future research is how to obtain useful long-term managerial decision information from results for all future uncertainty states. First year’s optimal decisions under uncertainty are easily obtained and based on long-term optimal decisions. However, extracting these long-term optimal decisions from the results is hard because the inclusion of uncertainty provides an exponentially increasing number of possible paths with their own probabilities. Instead of a decision tree type of visualization the current research proposes to investigate methods which provide a more compact representation of the optimization problem such as influence diagrams, and their results (Lander and Pinches 1998, Maier et al. 2018). One more interesting direction is presented by McGregor et al. (2017) who developed a visual interface for a specific MDP model.

Conclusions

Huge capital expenditures are involved in the renewal of ageing infrastructure. Emphasis is put on lifetime extension of infrastructure and optimizing a sequence of possible intervention strategies before renewal. Such optimization is challenging as infrastructure is subject to asset deterioration, structural failure and price uncertainties.

Current maintenance and replacement optimization models generally ignore price uncertainty despite researchers stressing its importance.

The current study developed an approach to include multiple price uncertainties in maintenance and replacement optimization for repairable infrastructure assets which occasionally fail and are allowed to fail. This model includes the flexibility to choose between multiple successive intervention strategies subject to asset degradation and structural failure.

State explosion is prevented by combining multiple price uncertainties into portfolios and exploiting the convenient characteristics of recombining binomial lattices. Whereas structural failure is modeled by interference theory, asset degradation follows a time-variant failure rate obtained expert judgment or trend testing, and transition probabilities are derived from available price data.

The model is demonstrated on an existing infrastructure case study with intervention strategies to maintain, overhaul, major overhaul and renew. The results indicate that price uncertainty indeed influences the first year’s optimal decision-making.

Hence, the current research validates the observations of previous researchers regarding the importance of addressing price uncertainty with respect to maintenance and replacement decisions. The forecast and inclusion of price uncertainty in current maintenance and optimization modeling deserve more attention in further research and professional practice.

The main managerial implication of the current research is that price uncertainty may influence optimal decision-making on maintenance and renewal. The approach developed benefits infrastructure asset owners and service contractors in accurate, short- and midterm decision-making under uncertainty while taking long-term optimal decision-making into account. Accurate short- and mid-term decision-making and planning is very important for capital intensive infrastructure with long design lives after renewal.

However, the current model is not without limitations. Although the results contain all future optimal decisions, this long-term planning information is difficult to extract from the results because of the many uncertainty states. As future research, the current research proposes to develop methods for a more compact visualization of results. Other directions for further research are method to reduce state explosion and inclusion of a probabilistic approach to asset deterioration modeling.

Acknowledgements

We owe much gratitude to Wouter van Vuuren, Ron van de Wiel, Wim Milder, Dert van Ree, and their colleagues from Water Authority Rivierenland for their valuable input: a case study with cost data, failure data and intervention strategies, and for their verification of the usefulness of the results.

Disclosure statement

No potential conflict of interest is reported by the authors.

Data availability statement

The data of construction prices that support the findings of this study are openly available in CBS Statline at https://opendata.cbs.nl/statline/#/CBS/en/dataset/81139eng/table?ts=1547989906393

The data on maintenance and energy prices that support the findings of this study are available at https://www.crow.nl/publicaties/indexen-risicoregelingen-gww.