Overview
A multi-GPU acceleration strategy has been developed for the ViCar3D sharp-interface immersed boundary solver. This framework aims to address the challenges associated with simulating flows involving complex moving boundaries, specifically focusing on the irregular data structures, dynamic sparsity, and discontinuous pressure operators inherent in these methods. The implementation is fully GPU-native and leverages a hybrid OpenACC/CUDA programming model.
Research Context
Sharp-interface immersed boundary methods offer a versatile framework for simulating fluid flows that interact with complex moving boundaries. However, their computational demands are significant, particularly in scenarios requiring high efficiency. Key challenges identified for efficient multi-GPU implementation include the irregular data structures, dynamically changing sparsity patterns, and discontinuous nature of pressure operators within these methods. The work specifically targets the ViCar3D solver, seeking to enhance its performance and scalability on multi-GPU systems.
Approach
The acceleration of the ViCar3D solver employs a combination of specific computational strategies:
- A hybrid OpenACC/CUDA programming model is utilized.
- A matrix decomposition technique is implemented to isolate dynamically changing immersed-boundary operators.
- GPU-optimized geometric kernels are integrated.
- Asynchronous dual-stream sparse matrix-vector execution is employed.
- A symmetric-memory-based sparse communication strategy is incorporated.
For the pressure Poisson equation, a GPU-oriented iterative solver was developed. This solver uses a preconditioned stabilized bi-conjugate gradient (BiCGStab) method, designed for efficient treatment of the pressure Poisson equation on highly stretched Cartesian grids.
Findings
The resulting framework demonstrated that it preserves the accuracy and versatility of the original ViCar3D solver. The multi-GPU implementation delivered substantial acceleration and scalability. These improvements were observed in simulations involving complex moving and deforming immersed boundaries.
Why This Matters
The substantial acceleration and scalability achieved extend the applicability of sharp-interface immersed boundary methods. This enables their use in extreme-scale computational fluid dynamics scenarios, which previously faced significant computational hurdles due to the challenges posed by complex moving and deforming immersed boundaries.