Long-term adaption decisions via fully and partially observable Markov decision processes

Abstract

Long-term decisions, such as the design of infrastructure systems and buildings or the planning of risk mitigation measures, should be made in consideration of the uncertain future. The initial design of a system determines its flexibility, i.e. its ability to cope with potential future changes. Increasing flexibility is generally considered to be a good approach to dealing with future uncertainty, such as climate change uncertainty, but its effects have not been systematically investigated. We propose the use of Markov Decision Processes combined with Influence Diagrams to solve adaptation planning problems. This framework can identify the optimal system type and capacity and determine the value of flexibility. It is here applied to two numerical examples: Planning of a waste water treatment plant under uncertainty in future population growth and planning of a flood protection system under uncertain climate change scenarios. Based on these idealized examples, it is shown that for flexible systems a lower initial capacity of the system is recommendable, while for inflexible systems a conservative design (with high safety factors) should be applied. The value of flexibility is shown to be high when significant learning is expected in the future, i.e. if information gathered in the future significantly reduces uncertainty.

Keywords:

• Decision-making under uncertainty
• infrastructure systems
• Bayesian updating
• Markov decision processes
• climate change
• flexibility
• changeability
• real options
• cost–benefit analysis

Notes

1. We use a discrete representation of the random variables (the demand θ_i). However, the framework can be extended to continuous RVs. In such case, the summation over the states of θ_i+1 is replaced by an integral over the domain of θ_i+1 and the probability mass functions and p(θ₁) are replaced with probability density functions and f(θ₁).