Second-order adjoints for solving PDE-constrained optimization problems

Abstract

Inverse problems are of the utmost importance in many fields of science and engineering. In the variational approach, inverse problems are formulated as partial differential equation-constrained optimization problems, where the optimal estimate of the uncertain parameters is the minimizer of a certain cost functional subject to the constraints posed by the model equations. The numerical solution of such optimization problems requires the computation of derivatives of the model output with respect to model parameters. The first-order derivatives of a cost functional (defined on the model output) with respect to a large number of model parameters can be calculated efficiently through first-order adjoint (FOA) sensitivity analysis. Second-order adjoint (SOA) models give second derivative information in the form of matrix–vector products between the Hessian of the cost functional and user-defined vectors. Traditionally, the construction of second-order derivatives for large-scale models has been considered too costly. Consequently, data assimilation applications employ optimization algorithms that use only first-order derivative information, such as nonlinear conjugate gradients and quasi-Newton methods.

In this paper, we discuss the mathematical foundations of SOA sensitivity analysis and show that it provides an efficient approach to obtain Hessian-vector products. We study the benefits of using second-order information in the numerical optimization process for data assimilation applications. The numerical studies are performed in a twin experiment setting with a two-dimensional shallow water model. Different scenarios are considered with different discretization approaches, observation sets, and noise levels. Optimization algorithms that employ second-order derivatives are tested against widely used methods that require only first-order derivatives. Conclusions are drawn regarding the potential benefits and the limitations of using high-order information in large-scale data assimilation problems.

Keywords:

• inverse problems
• second-order adjoints
• stiff equations
• sensitivity analysis
• shallow water equations
• data assimilation
• numerical optimization
• PDE-constrained optimization

Acknowledgements

This work was supported by the National Science Foundation through the awards NSF DMS-0915047, NSF CCF-0635194, NSF CCF-0916493 and NSF OCI-0904397, by NASA through the award AIST-2005, and by the Houston Advanced Research Center through the award H-98/2008. The authors thank Prof. Florian Potra for suggesting the Moser–Hald method.