A parallel computational model for sensitivity analysis in optimization for robustness

Abstract

We present an efficient method for computing, in a parallel distributed environment, gradients of functionals depending on the solution sensitivities of dynamical systems described by ordinary differential equations. This work was motivated by the need to compute cost function gradients for dynamically constrained optimization for robustness, i.e. the problem of finding the model parameters of a given dynamical system that lead to minimum solution sensitivity. The proposed approach for this particular second-order sensi-tivity problem falls into the class of the so-called ‘adjoint-over-forward’ methods and is based on solving the continuous forward sensitivity equations (for evaluating the cost functional) and their continuous adjoint systems (for evaluation of the gradient of the cost functional). Efficiency is ensured by: (i) a suitable distribution over processors of the individual sensitivity systems, both forward and adjoint so that inter-process communication is minimized; and (ii) use of an iterative solver combined with a block-diagonal preconditioner for the solution of the linear systems arising in the implicit integration of the resulting ordinary differential equations (ODE) systems. The proposed algorithm was implemented as an extension of the cvodes solver in sundials [P. Brown, K. Grant, A. Hindmarsh, S. Lee, R.S.D. Shumaker, and C. Woodward. SUNDIALS: SUite of Nonlinear and DIfferential/ALgebraic equation Solvers, ACM Trans. Math. Software, 31(3) (2005), pp. 363–396], but it can be used in conjunction with any sensitivity-enabled ODE integrator that provides adjoint sensitivity capabilities and (if based on implicit integration methods) support for iterative linear algebra.

Keywords:

• sensitivity analysis
• adjoint method
• parallel distributed computing

AMS Subject Classification :

• 65L05
• 65Y05
• 65L99

Acknowledgements

This work was performed under the auspices of the US Department of Energy by Lawrence Livermore National Laboratory under contract no. DE-AC52-07NA27344.

Notes

Since (19) changes type at μ=0, any discretization must be such that there are no points at μ=0. In our case, this is ensured by enforcing n _μ to be odd.

In all fairness, it should be noted that this is a self-imposed limitation. Indeed, in order to minimize the impact of the integration of the adjoint problems (17) and (18) on the parallel speedup results and hence more accurately measure the parallel performance of the as2 code and not that of the cvodes solver, we have ensured that no checkpointing will be required. Since the two-pass checkpointing scheme employed in cvodes [16 practically attempts to strike a balance between computation and memory requirements, with dense enough checkpointing, Problem 2 could also be solved on fewer processors.