Solution of the Neutron Transport Equation on Unstructured Grids Using the Parallel Block Jacobi-Integral Transport Matrix Method via the Novel Green’s Function ITMM Construction Algorithm on Massively Parallel Computer Systems

Abstract

The Parallel Block Jacobi (PBJ) spatial domain decomposition is well suited for implementation on massively parallel computers to solve the neutron transport equation on unstructured grids due to the simple scheduling policy that arises from the PBJ’s iterative asynchronicity. The Parallel Block Jacobi-Integral Transport Matrix Method (PBJ-ITMM) is an iterative method that utilizes the PBJ decomposition and resolves local within-group scattering in a single iteration, but requires a matrix-vector iterative solution. This work details the development, implementation, and testing of the novel Green’s Function ITMM Construction (GFIC) algorithm. The GFIC constructs the matrices required for the PBJ-ITMM’s iterative solution on unstructured grids, utilizing the physical interpretation of these matrices as discretized response functions to create a local problem with a Green’s Function–like source. Conducting a set of mesh sweeps over all angles on this local problem yields the ITMM matrix elements. On unstructured grids, this approach utilizes the kernel calculation and fundamental solution algorithm present in an existing transport code, thus avoiding reimplementation of code functionality. Using the GFIC, the PBJ-ITMM is implemented in THOR, a tetrahedral mesh transport code, along with the Inexact Parallel Block Jacobi (IPBJ) method for performance comparison. This comparison involves strong and weak scaling studies of the Godiva and C5G7 benchmark problems using up to 32 768 processors. These studies establish that the PBJ-ITMM executes faster than the IPBJ when the number of cells per subdomain falls below a problem-dependent threshold, ~128 cells for Godiva, >256 cells for C5G7. The largest problem tested, comprising more than 6.8 billion unknowns, solves in <30 min with the IPBJ and <20 min with the PBJ-ITMM, using 32 768 processors. These results demonstrate the PBJ-ITMM as a viable approach for solving neutron transport problems on unstructured grids using massively parallel computers. Additionally, this study illustrates the range of number of cells per subdomain over which this method is favorable.

Keywords:

• Neutron transport
• massively parallel
• unstructured grids
• Parallel Block Jacobi

Acknowledgments

The work of the author is based on work supported by the U.S. Department of Energy National Nuclear Security Administration under award number DE-NA0002576.

This paper was prepared as an account of work sponsored by an agency of the U.S. government. Neither the U.S. government nor any agency thereof, nor any of their employees, makes any warranty, express or implied, or assumes any legal liability or responsibility for the accuracy, completeness, or usefulness of any information, apparatus, product, or process disclosed, or represents that its use would not infringe privately owned rights. Reference herein to any specific commercial product, process, or service by trade name, trademark, endorsement, recommendation, or favoring by the U.S. government or any agency thereof. The views and opinions of authors expressed herein do not necessarily state or reflect those of the U.S. government or any agency thereof.

This research made use Idaho National Laboratory computing resources that are supported by the Office of Nuclear Energy of the U.S. Department of Energy and the Nuclear Science User Facilities under contract number DE-AC07-05ID14517.

The authors would like to personally acknowledge Drs. Sebastian Schunert and Joe Zerr for their indispensable contribution to this work. We sincerely appreciate their willingness to obtain computational resources and to assist in any way possible.