Overview
PhysicsFormer is a proposed Transformer-based physics-informed neural network (PINN) designed for the simulation of complex fluid flows. This framework deviates from conventional multilayer perceptron-based PINNs by incorporating an encoder-decoder multi-head attention mechanism. This architectural choice aims to capture long-range temporal dependencies within the fluid dynamics data and enhance spatiotemporal information propagation.
The framework also utilizes pseudo-sequential spatiotemporal representations combined with a dynamics-weighted loss formulation. This approach was implemented to improve convergence, stability, and predictive accuracy in fluid flow simulations. PhysicsFormer is characterized by a lightweight architecture and a parallel learning strategy, which contribute to faster training and reduced computational costs when compared to existing Transformer-based PINN models.
Research Context
Traditional computational fluid dynamics (CFD) methods and existing physics-informed neural networks (PINNs) encounter challenges in specific scenarios. These challenges often include high computational cost, sensitivity to mesh configurations, and reduced accuracy, particularly when applied to strongly nonlinear and time-dependent fluid flows. The development of PhysicsFormer seeks to address these noted limitations within the domain of fluid simulations.
Approach
The PhysicsFormer framework integrates a Transformer-based architecture configured with an encoder-decoder multi-head attention mechanism. This mechanism is specifically engineered to identify and exploit long-range temporal dependencies inherent in fluid flow data, thereby facilitating improved spatiotemporal information propagation. Instead of relying solely on multilayer perceptron components, the model incorporates pseudo-sequential spatiotemporal representations. A key element of the framework is the dynamics-weighted loss formulation, which is intended to optimize the training process by improving convergence, stability, and the overall predictive accuracy of the model.
The design principles emphasize a lightweight architecture concurrent with a parallel learning strategy. This combination was implemented to achieve computational efficiency, leading to faster training times and lower computational resource requirements compared to other Transformer-based PINN models.
Findings
- PhysicsFormer was demonstrated on several fluid dynamics problems: the convection equation, Burgers' equation, and lid-driven cavity flow at $Re=100$.
- The framework was also applied to inverse Navier–Stokes and flow reconstruction problems for flow past a circular cylinder at $Re=100$ and $Re=3900$.
- For the inverse Navier–Stokes problem at $Re=100$, PhysicsFormer simultaneously reconstructed the flow field and identified governing equation parameters. This process yielded nearly 0% absolute error under both clean and noisy data conditions.
- In the high-Reynolds-number case of flow past a circular cylinder at $Re=3900$, PhysicsFormer accurately reconstructed the velocity and pressure fields. This reconstruction utilized only 25 spatial measurements per snapshot across 100 temporal snapshots.
- The obtained results suggest that PhysicsFormer provides an accurate, robust, and computationally efficient framework for complex time-dependent fluid flow problems.
Why This Matters
The development of PhysicsFormer offers a method for addressing the computational cost, mesh sensitivity, and reduced accuracy often associated with traditional computational fluid dynamics and existing physics-informed neural networks. Its ability to accurately reconstruct flow fields and identify governing parameters, even under challenging conditions such as noisy data and high Reynolds numbers, indicates a potential to improve the efficiency and reliability of complex fluid flow simulations.