Towards massive parallel fluid flow simulations in computational engineering

The trend for computers to get faster and faster means it is now possible to simulate many larger problems from an engineering-based domain, such as complex indoor air flow scenarios driven by buoyancy, which were deemed unsolvable a decade ago. Using a massive parallel approach on a supercomputer poses some challenges which have to be tackled before initial results can be achieved, however. This work presents a simulation pipeline which is able to work efficiently on massive parallel systems such as SuperMUC, one of Germany's national supercomputers. After describing the mathematical formulation of an incompressible Newtonian fluid flow based on the Navier-Stokes equations, the standard numerical methods such as the finite difference and finite volume based approaches are discussed. The newly introduced data structure forms the backbone of the complete system and consists of an adaptive, hierarchic, block-structured, orthogonal, Cartesian grid structure. The communication routines for data exchange in the hierarchical structure are highlighted. Furthermore, a load balancing concept, which is essential for massive parallel approaches, is analysed and described in detail. A pressure correction method based on Chorin's projection method is applied to decouple the velocity and pressure fields, resulting in a Poisson equation for the pressure which has to be solved in every time step. An efficient method based on a multi-grid-like solver and integrated into the data structure is presented, and accuracy as well as performance measurements of the data structure are presented for up to 80 billion pressure unknowns solved on SuperMUC. Furthermore, an interactive visualisation technique is introduced, which is capable of selecting only a small detail of the entire domain in fine resolution or the complete overview in a coarser resolution, in order to handle the massive amount of data generated by the type of computation mentioned above. Standard benchmarks are presented, which show the excellent agreement of the simulation results and literature values for the most common examples, such as a lid-driven cavity, a channel flow, a von Kármán vortex street, heated side walls, and a Rayleigh-Bénard convection. Last but not least, a complex example of a human manikin in a test room is simulated by a zonal computation coupled with the CFD simulation presented in order to obtain a detailed snapshot of the velocity distribution in the room together with a detailed thermoregulation model for evaluating the temperatures on the surface of the manikin. This work highlights possibilities to efficiently combine methods from computer science – especially from the field of high-performance computing – with problems from engineering-based domains in order to leverage and facilitate the treatment of complex problems which have not been able to be solved directly until now. The synergy effects between the two disciplines are obvious and promising. On the engineering side, a deeper insight into the problem itself can be gained, for a coupled thermal simulation with a human manikin model, for example. Furthermore, on the computer science side, the methodological concepts such as the proposed multi-grid-like solver concept can be expanded and optimised in order to be used efficiently in engineering-based applications.