Abstract
In this work, we take a modern high-resolution finite-volume scheme for solving the rotational shallow-water equations and extend it with features required to run real-world ocean simulations. Our contributions include a spatially varying north vector and Coriolis term required for large scale domains, moving wet-dry fronts, a static land mask, bottom shear stress, wind forcing, boundary conditions for nesting in a global model, and an efficient model reformulation that makes it well-suited for massively parallel implementations. Our model order is verified using a grid convergence test, and we show numerical experiments using three different sections along the coast of Norway based on data originating from operational forecasts run at the Norwegian Meteorological Institute. Our simulation framework shows perfect weak scaling on a modern P100 GPU, and is capable of providing tidal wave forecasts that are very close to the operational model at a fraction of the cost. All source code and data used in this work are publicly available under open licenses.
Acknowledgements
This research has mainly been funded by the Research Council of Norway under grant number 250935 (GPU Ocean), and partly by grant number 310515 (Havvarsel). The GPU Ocean core project team consists of Göran Broström, Kai Christensen, Knut-Andreas Lie, and Martin Lilleeng Sætra, and the authors are sincerely grateful for their collaboration and discussions, which have influenced and enabled this work. The authors also thank Øyvind Sætra and Jon Albretsen for feedback on drafts of the manuscript in addition to the thorough comments from the reviewers. The GPU Ocean project has received support in form of compute time on UNINETT Sigma2 - the National Infrastructure for High Performance Computing and Data Storage in Norway under project number nn9550k. The authors declare that we have no competing interests related to this work.
Notes
1 The source code presented in this paper, along with Jupyter Notebooks (Kluyver et al. 2016) used to create the plots and results, is available as open source under GPL version 3, and has been published on Zenodo under DOI: 10.5281/zenodo.4056327. We have used the NorKyst-800 forecast for 2019-07-16 from the Norwegian Meteorological Institute in this work, available on https://thredds.met.no/thredds/dodsC/fou-hi/norkyst800m-1h/NorKyst-800m_ZDEPTHS_his.an.2019071600.nc. The observed sea-surface levels are obtained from Se Havnivå, https://www.kartverket.no/sehavniva/, provided by the Norwegian Mapping Authority, Hydrographic Service. The data sets are available under the Creative Commons 4.0 BY International license as described on https://www.met.no/en/free-meteorological-data/Licensing-and-crediting and https://www.kartverket.no/en/data/Terms-of-use/.
2 The spatial discretization of the numerical fluxes in (Chertock et al., Citation2018) reduces to the fluxes presented in (Kurganov and Levy, Citation2002) when friction terms and the Coriolis parameter is set to zero.
3 The Nvidia GeForce GTX780, used as an example of a consumer GPU in this study, has a performance of 4.1 teraFLOPS in single precision, and 0.17 teraFLOPS in double precision. The recent Nvidia TITAN RTX has a performance of 16.3 teraFLOPS in single precision, and 0.51 teraFLOPS in double precision. The professional GPU used in this study, the Nvidia Tesla P100, has a performance of 9.5 teraFLOP in single precision, and 4.8 teraFLOPS in double precision. Other consumer grade and professional grade GPUs follow the same trend.
4 Also known as a sponge layer.
5 Textures on GPUs are originally designed to hold the colour of one pixel as the colour channels red, green, blue, and alpha, hence float4.
6 Grey dilation computes the maximum value of a small neighbourhood (typically 3 × 3 cells) for each location.
7 Please note that the ROMS model is far more complex and can properly represent the ocean state. Our simplified model is only capable of capturing the barotropic dynamics and, e.g., not capable of creating long-term forecasts or capture salinity and temperature driven dynamics.